Effects of game-based learning in STEM education vary substantially across studies, with prediction intervals spanning trivial-to-negative through very strong effects¶

Claim · All claims
Evidence · 1 study · quant-synthesis r? · q3
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Between-study heterogeneity is large (I2 76%-92%) and true effects disperse widely around the mean, including negative effects in some populations. → Michaela Arztmann 2023

Evidence¶

Michaela Arztmann 2023¶

Michaela Arztmann, Lisette Hornstra, Johan Jeuring & Liesbeth Kester (2023). Effects of games in STEM education: a meta-analysis on the moderating role of student background characteristics. Studies in Science Education, 59:1, 109-145. https://doi.org/10.1080/03057267.2022.2057732

q3 · i? · quant-synthesis · r?

Random-effects meta-analysis reporting heterogeneity statistics: I2 ranges from 76% to 92% across the three outcomes, and the article states "the dispersion of effects around this mean is substantial", with prediction intervals of −0.11 to +1.12 for motivation and +0.08 to +1.78 for behaviour.

"For cognition, the 95% prediction interval ranges from −0.38 to +1.76. This means, that in the population represented by the included studies, the true effect size in 95% of cases will fall somewhere in this range"

Discussion¶

Related Claims¶

• School level shows a marginal moderating effect on motivational outcomes, with secondary students experiencing a lower motivating effect than primary students
• Design parameters for preschool outcomes show substantial heterogeneity across populations, as reflected in wide 95% prediction intervals — related
• Effects of AI literacy interventions vary considerably across contexts, with most variance between studies (71.361%) and a wide prediction interval — related
• The positive GenAI writing effect is robust across dependence-aware, sensitivity, and risk-of-bias specifications, but the wide prediction interval spans near-zero to large effects — a narrower finding that bears on this claim
• School level moderates cognitive effects of game-based learning in STEM: games are less effective for secondary than primary school students — related
• Game-based learning in primary and early secondary STEM education shows moderate positive effects on cognition, motivation, and behaviour compared to traditional classrooms — related
• Effect sizes vary widely between studies and especially between comparisons within studies — related
• Effect sizes across shared text reading studies show substantial true heterogeneity (I2 = 80.09) — related
• Effect sizes across studies show substantial heterogeneity, with wide prediction intervals spanning large negative to large positive true effects — related
• Effect sizes for gender differences in L2 achievement are highly heterogeneous across studies (I² = 89.82%) — related