Among live-evaluation RL-in-education papers reporting significant differences, a majority reported at least one intermediate (n = 5) or large (n = 6) Cohen's d effect on post-test learning outcomes¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Of 15 papers suitable for effect-size overview, a majority of publications finding significant differences reported at least one intermediate (n = 5) or large effect (n = 6). → Anna Riedmann 2025

Evidence¶

Anna Riedmann 2025¶

Anna Riedmann, Philipp Schaper, Birgit Lugrin. (2025). Reinforcement Learning in Education: A Systematic Literature Review. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00494-6

q2 · i? · quant-synthesis · r2

The review's effect-size analysis (Fig. 9) covers Cohen's d and 95% confidence intervals for post-test score comparisons on learning-related outcomes, restricted to "live" evaluations; for seven papers, effect sizes were calculated from descriptive values using Lenhard and Lenhard's online calculator.

"We identified 15 papers to be suitable for our overview of effect sizes (see Fig. 9). It is notable that a majority of publications finding significant differences reported at least one intermediate (n = 5) or large effect (n = 6)."

Discussion¶

Related Claims¶

• Content-related adaptation (n = 53 papers) outnumbers guidance-related adaptation (n = 36) in RL-based educational applications — related
• Among RL-in-education papers testing statistical significance, 18 of 35 report at least one RL policy significantly outperforming all baselines, while over half of all reviewed papers report no statistical analysis — related
• RL-based educational evaluations split across live evaluation (n = 41), real interaction data (n = 23), and simulated users (n = 24) — related
• Across 15 school meditation studies, 61% of calculable effect sizes were statistically significant, with most effects small — related
• Among RL papers statistically outperforming baselines, normalized learning gain was the most common effective reward source, used in over half of the papers (n = 10) — related