Educational level does not significantly moderate Gen-AI's effect on HOT, though K-12 subgroups show a larger effect (ES = 0.857) than post-secondary (ES = 0.539)¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3 · i2 medium
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 The between-groups difference for educational level did not reach significance (Q = 3.353, p = 0.067), but subgroup effects differ: K-12 ES = 0.857 versus post-secondary ES = 0.539. → Zhao Yan 2025

Evidence¶

Zhao Yan 2025¶

Zhao Yan, Yue Yuhe, Sun Zhonghua, Jiang Qiang, Li Gangsheng. (2025). Does Generative Artificial Intelligence Improve Students’ Higher-Order Thinking? A Meta-Analysis Based on 29 Experiments and Quasi-Experiments. Journal of Intelligence. https://doi.org/10.3390/jintelligence13120160

q3 · i2 · quant-synthesis · r3

Moderator analysis (Table 5) of educational stage: K-12 N = 13, ES = 0.857; post-secondary N = 46, ES = 0.539 (Table 5 prints 0.539; the text narrative reports 0.593), both p < 0.001, but the omnibus between-groups test was non-significant.

"the between-groups difference did not reach statistical significance (Q = 3.353, p = 0.067), suggesting that educational level did not constitute a statistically significant moderator in the relationship between Gen-AI and students’ HOT"

Discussion¶

Related Claims¶

• Instructional method does not significantly moderate Gen-AI's effect on HOT, though project-based learning shows the largest subgroup effect (ES = 0.717) and lecture-based instruction a small non-significant effect (ES = 0.396) — related
• Intervention duration significantly moderates Gen-AI's effect on HOT, with the strongest effects at 8–16 weeks (ES = 0.759) and weaker effects for shorter (0.494) and longer (0.372) interventions — related
• Self-regulated learning ability significantly moderates Gen-AI's effect on HOT: high-SRL students show ES = 0.863 versus ES = 0.284 for low-SRL students — related
• Gen-AI has a moderate positive overall effect on students' higher-order thinking (g = 0.609) across 29 experimental and quasi-experimental studies — related
• Gen-AI's effect on HOT is strongest for problem-solving ability (g = 0.745), followed by critical thinking (g = 0.691), with a smaller effect on creativity (g = 0.444) — related
• The motivational effect of GenAI is larger in higher education than in primary and secondary education (exploratory) — related
• Testing effects hold in classrooms (g = 0.67) as in laboratories (g = 0.62), and are largest for secondary students (g = 0.83) — related
• Technology effects on TEFL outcomes trend upward across educational levels (primary g = 0.29, secondary g = 0.35, tertiary g = 0.44), though between-level differences are not statistically significant — related