In peg-solitaire simulations with known true abilities, the RLMM attains lower RMSE of log βj than the MDP-MM on all four original boards¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Across the four peg-solitaire boards from LaMar (2018), the RLMM recovered simulated person parameters more accurately than the MDP-MM, with error reductions from roughly one-third to more than two-thirds. → Wenqian Xu 2026
Evidence¶
Wenqian Xu 2026¶
Wenqian Xu, Feng Ji. (2026). Reinforcement Learning Measurement Model. arXiv. https://arxiv.org/abs/2605.09305 doi:10.48550/arxiv.2605.09305
q2 · i? · design · r2
Simulation study on the four peg-solitaire boards of LaMar (2018) with 200 simulated students completing 50 games per board; printed RMSE values include 0.542 vs 0.368 (tiny cross) and 0.403 vs 0.099 (diamond), showing "relative reductions in error". Scatterplots showed estimates close to the diagonal.
"Across all four tasks, the RLMM attains lower RMSE than the MDP-MM, with relative reductions in error from roughly one-third on the smallest board to more than two-thirds on the diamond board."
Discussion¶
Related Claims¶
- Influence-function diagnostics identify steps 3–8 of the Big-L peg-solitaire board as the most consequential region, with the lowest-β participants showing substantially larger absolute influence values there — related
- The RLMM recovers ability parameters on peg-solitaire boards with up to more than 23 million states, where tabular MDP-MM estimation is impractical — related
- RLMM runtime stays within approximately 15–20 seconds across the four peg-solitaire boards while MDP-MM runtime grows with task complexity, reaching a 19.1× speedup on the diamond board — related
- In AQUALAB, estimated β is ordered across episode completion status (means 1.181, 1.506, 2.343; Kruskal–Wallis p = 8.62×10−29) and higher-β quintiles complete more content more efficiently
- In AQUALAB gameplay logs, the estimated RLMM person parameter is positively associated with cumulative trajectory reward (Pearson 0.5842, Spearman 0.5760)
- Hint Factory MDP-generated hints gave correct next-step hints over 80% of the time and a pilot showed students solved more logic proofs with hints — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
rlmm-lower-rmse-than-mdp-mm-peg-solitaire |
title |
In peg-solitaire simulations with known true abilities, the RLMM attains lower RMSE of log βj than the MDP-MM on all four original boards |
description |
In peg-solitaire simulations with known true abilities, the RLMM attains lower RMSE of log βj than the MDP-MM on all four original boards |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — wenqian-xu-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.