Among RL papers statistically outperforming baselines, normalized learning gain was the most common effective reward source, used in over half of the papers (n = 10)¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Learning gain, particularly normalized learning gain, appeared to be an effective reward source and was used in over half of the 18 papers with statistically outperforming RL approaches (n = 10). → Anna Riedmann 2025

Evidence¶

Anna Riedmann 2025¶

Anna Riedmann, Philipp Schaper, Birgit Lugrin. (2025). Reinforcement Learning in Education: A Systematic Literature Review. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00494-6

q2 · i? · review · r2

The review assessed reward-function elements of the 18 papers whose RL approaches statistically outperformed one or more baselines. Accuracy (often combined with time or consecutive incorrect attempts) appeared in three approaches; two defined rewards from learning gain plus engagement, one solely on engagement.

"Learning gain, particularly normalized learning gain, which quantifies students' progress regardless of their initial competence, appeared to be an effective reward source and was used in over half of the papers (n = 10)."

Discussion¶

Related Claims¶

• Among RL-in-education papers testing statistical significance, 18 of 35 report at least one RL policy significantly outperforming all baselines, while over half of all reviewed papers report no statistical analysis — related
• Over eight weeks, university learners receiving reinforcement learning-optimized oral practice sequencing attained normalized learning gains approximately 2.2 times higher than learners following fixed curricula — related
• Model-free methods dominate RL in education (n = 72 of 89 papers), with value-based approaches most common (n = 51) and Q-learning the most frequently mentioned algorithm (n = 28) — related
• Reinforcement learning in adaptive educational systems faces sample inefficiency and reward-design instability, motivating hybrid RL-ML approaches — related
• Among live-evaluation RL-in-education papers reporting significant differences, a majority reported at least one intermediate (n = 5) or large (n = 6) Cohen's d effect on post-test learning outcomes — related
• An analytical chapter identifies reward hacking, engagement optimization over learning, and scaffolding dependency as risks of RL policies with poorly specified rewards — related
• Reinforcement learning operationalizes motivational constructs by representing motivation in the reward function — a broader claim this one bears on