Gen-AI has a moderate positive overall effect on students' higher-order thinking (g = 0.609) across 29 experimental and quasi-experimental studies¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3 · i2 medium
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 Meta-analysis of 59 effect sizes from 29 studies yields an overall Hedges's g of 0.609 for Gen-AI's effect on students' HOT. → Zhao Yan 2025

Evidence¶

Zhao Yan 2025¶

Zhao Yan, Yue Yuhe, Sun Zhonghua, Jiang Qiang, Li Gangsheng. (2025). Does Generative Artificial Intelligence Improve Students’ Higher-Order Thinking? A Meta-Analysis Based on 29 Experiments and Quasi-Experiments. Journal of Intelligence. https://doi.org/10.3390/jintelligence13120160

q3 · i2 · quant-synthesis · r3

Random-effects meta-analysis of 59 effect sizes from 29 experimental and quasi-experimental studies, using Hedges's g. The pooled estimate was "an overall effect size of 0.609", in the moderate range (0.2–0.5 per Cohen's guidelines is moderate; the authors label 0.5–0.8 significant). Heterogeneity was high (I² = 77.273%), so a random-effects model was used.

"the analysis yielded an overall effect size of 0.609, indicating that Gen-AI has a moderately significant and statistically meaningful impact on students’ HOT abilities"

Discussion¶

Related Claims¶

• Meta-reviews reach contradictory conclusions on whether AI reliance harms or helps critical thinking, with substantial design limitations limiting generalizability — the general claim this finding bears on, with the studies behind it
• AI-supported instruction has a moderate positive effect on student achievement in STEM education (g = 0.670 under a random effects model) — related
• Educational level does not significantly moderate Gen-AI's effect on HOT, though K-12 subgroups show a larger effect (ES = 0.857) than post-secondary (ES = 0.539) — related
• Gen-AI's effect on HOT is strongest for problem-solving ability (g = 0.745), followed by critical thinking (g = 0.691), with a smaller effect on creativity (g = 0.444) — a narrower finding that bears on this claim
• Intervention duration significantly moderates Gen-AI's effect on HOT, with the strongest effects at 8–16 weeks (ES = 0.759) and weaker effects for shorter (0.494) and longer (0.372) interventions — related
• Instructional method does not significantly moderate Gen-AI's effect on HOT, though project-based learning shows the largest subgroup effect (ES = 0.717) and lecture-based instruction a small non-significant effect (ES = 0.396) — related
• Self-regulated learning ability significantly moderates Gen-AI's effect on HOT: high-SRL students show ES = 0.863 versus ES = 0.284 for low-SRL students — related
• GenAI-supported learning is associated with higher learning motivation on average (g = 0.764), but with very high between-study heterogeneity (I2 = 93.7%) — related
• AI literacy interventions show a large, statistically significant overall positive effect on student AI literacy outcomes (g = 0.837) — related
• Study quality moderated estimated effects: quasi-experimental designs yielded higher effect size estimates than random designs, and strong validity reporting was associated with higher estimates — related
• AI-enabled personalized STEM education yields a statistically significant overall effect of g = 0.455, which its authors class as small, compared with traditional non-AI instruction in K-12 schools — related
• Prior meta-analytic evidence indicates chatbot technology has a substantial positive effect on overall learning outcomes and knowledge retention, but not on critical thinking, engagement, or motivation — reports the opposite
• AI applications contributed to STEM students' higher-order thinking, including problem-solving, computational thinking, and self-regulated learning — a narrower finding that bears on this claim
• Technology type did not significantly moderate effects on metacognitive outcomes, though generative-AI interventions showed a descriptively larger estimate — related
• Technology effects on TEFL outcomes trend upward across educational levels (primary g = 0.29, secondary g = 0.35, tertiary g = 0.44), though between-level differences are not statistically significant — related
• GenAI-supported writing instruction yields a statistically significant positive pooled effect on L2/EFL writing performance (g = 0.80) with very high heterogeneity — related
• GenAI shows a dual-mechanism profile: meta-analytic learning benefits coexist with dependence-related costs to critical thinking — related
• Collaborative problem-solving significantly improves students' critical thinking overall (ES = 0.82) — related