Among RL-in-education papers testing statistical significance, 18 of 35 report at least one RL policy significantly outperforming all baselines, while over half of all reviewed papers report no statistical analysis¶
Claim · All claims
Evidence · 1 study · reviewr2·q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Of 35 papers testing for statistical significance, 18 found at least one RL policy significantly outperforming all baselines and two demonstrated a significant aptitude-treatment effect benefiting low-performing students. → Anna Riedmann 2025
q2 i? Over half of all included papers (n = 54) did not statistically analyze their results for significance, reporting only descriptive values or figures. → Anna Riedmann 2025
Evidence¶
Anna Riedmann 2025¶
Anna Riedmann, Philipp Schaper, Birgit Lugrin. (2025). Reinforcement Learning in Education: A Systematic Literature Review. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00494-6
q2 · i? · review · r2
Systematic review of 89 papers (PRISMA, three databases, 2000–2024) categorizing results into six significance levels. The review reports that "over half of all included papers (n = 54) did not statistically analyze their results for significance"; of the 35 that tested, 18 had an RL policy outperforming all baselines and two showed ATI effects.
"over half of all included papers (n = 54) did not statistically analyze their results for significance, specifying only descriptive values (e.g., means and standard deviations) or figures"
Discussion¶
Related Claims¶
- Among RL papers statistically outperforming baselines, normalized learning gain was the most common effective reward source, used in over half of the papers (n = 10) — related
- Among live-evaluation RL-in-education papers reporting significant differences, a majority reported at least one intermediate (n = 5) or large (n = 6) Cohen's d effect on post-test learning outcomes — related
- RL-based educational evaluations split across live evaluation (n = 41), real interaction data (n = 23), and simulated users (n = 24) — related
- Students with lower baseline performance (below the 50th percentile) showed significantly greater math achievement gains with ASSISTments than matched virtual peers — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
rl-education-significance-18-of-35-outperform |
title |
Among RL-in-education papers testing statistical significance, 18 of 35 report at least one RL policy significantly outperforming all baselines, while over half of all reviewed papers report no statistical analysis |
description |
Among RL-in-education papers testing statistical significance, 18 of 35 report at least one RL policy significantly outperforming all baselines, while over half of all reviewed papers report no statistical analysis |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
generated |
by process:wiki-ingest on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — anna-riedmann-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.