Across all three domains, trained StudentSim simulators outperform every baseline on guidance responsiveness (R), with the largest gap on open-ended guidance modes¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-17 (automated, 2 of 2 entries) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across chess, L2, and math, the trained simulators outperform every baseline on aggregate R, and the gap is largest on open-ended guidance modes. → Ke Yang 2026 q2 i? On the chess guidance-following test set, StudentSim reaches R=0.91 versus GPT-4o 0.77, GPT-5.4 0.72, and Maia2 0.27. → Ke Yang 2026 (2)

Evidence¶

Ke Yang 2026¶

Ke Yang, Chenglong Wang, Michel Galley, Chandan Singh, Jeevana Priya Inala, ChengXiang Zhai, Jianfeng Gao. (2026). StudentSim: Training LLM-based Student Simulators. https://arxiv.org/abs/2609.01591 doi:10.48550/arxiv.2609.01591

q2 · i? · design · r2

Guidance responsiveness results across the three domains: "the trained simulators outperform every baseline on aggregateR", with the authors noting the gap is largest where the tutor prompts reasoning through questions or principles rather than stating the correction.

"Across all three domains, the trained simulators outperform every baseline on aggregateR (Table 3); the gap is largest on the open-ended guidance modes, where the tutor prompts the student to reason toward the fix through questions or principles."

Ke Yang 2026 (2)¶

Ke Yang, Chenglong Wang, Michel Galley, Chandan Singh, Jeevana Priya Inala, ChengXiang Zhai, Jianfeng Gao. (2026). StudentSim: Training LLM-based Student Simulators. https://arxiv.org/abs/2609.01591

q2 · i? · design · r2

Chess guidance-following evaluation on held-out positions: "STUDENTSIMreachesR=0.91" versus GPT-4o, GPT-5.4, and Maia2; Maia2's score is a zero-guidance lower bound because it has no natural-language input pathway. No effect size is printed.

"Across the full guidance-following test set, STUDENTSIMreachesR=0.91, vs. GPT-4oR= 0.77, GPT-5.4R= 0.72, and Maia2R= 0.27."

Discussion¶

Related Claims¶

• Across chess, L2 writing, and math, trained StudentSim simulators outperform every reference baseline on behavioral fidelity (F) — related
• Using a trained StudentSim as the reward for chess tutor RL yields a tutor that expert human raters score best on accuracy, guidance quality, and personalization among the rewards studied — related
• A 4B per-student simulator solves a held-out Socratic-guidance chess case where prompted GPT-5.4 fails, indicating multi-turn training adds guidance-following capability beyond hint copying — a narrower finding that bears on this claim
• ProfileRL outperforms prompting and fine-tuning baselines on turn-level student simulation, with statistically significant gains on Acts, Correctness, and Errors — related