Both RL stages are needed: removing DPO and GRPO training causes the largest performance drop in student simulation¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Removing both DPO for the simulator and GRPO for the profile generator decreases Acts from 0.605 to 0.589, Corr. from 0.466 to 0.436, and Errors from 0.150 to 0.115. → Zhangqi Duan 2026 q2 i? History SFT, which removes all three components of the method, performs poorly, and prompting-only use of the profile performs worst. → Zhangqi Duan 2026

Evidence¶

Zhangqi Duan 2026¶

Zhangqi Duan, Shuyan Huang, Alexander Scarlatos, Jaewook Lee, Simon Woodhead, Andrew Lan. (2026). Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues. arXiv preprint. https://arxiv.org/abs/2605.30051 doi:10.48550/arxiv.2605.30051

q2 · i? · causal · r2

Ablation study on one data split (Table 2) varying training stages and profile inputs. The article concludes "SFT alone is not sufficient for faithful student simulation" and that the two optimization stages are complementary.

"Removing both DPO for dialogue simulator training and GRPO for profile generator training leads to the largest drop in performance, decreasing Acts from 0.605 to 0.589, Corr. from 0.466 to 0.436, and Errors from 0.150 to 0.115."

Discussion¶

Related Claims¶

• GRPO training shifts profiles from broad summaries toward diagnostic, behaviorally grounded representations that better support simulation — related
• Knowledge and behavior profile components provide complementary signals: removing either hurts metrics tied to the other — related
• ProfileRL outperforms prompting and fine-tuning baselines on turn-level student simulation, with statistically significant gains on Acts, Correctness, and Errors — related
• Removing the DPO objective reduces examination accuracy by 1.11 percentage points and degrades annotated pedagogical quality — related
• Reinforcement learning with a learned alignment reward substantially improves question-concept alignment after supervised fine-tuning, while answerability remains high throughout training — related
• SFT with simulated children's reading errors yields higher LM-PPL than other experiments, driven by brevity and post-story irrelevant content — related
• Federated aggregation is the primary driver of performance gains, contributing an average 11.99% improvement — related
• DAPO was selected over GRPO for the RL stage because its decoupled policy update gave more stable gradients and superior CDPK performance — related
• In the EduAgent framework, removing past cognitive states from memory significantly reduced simulation performance — related
• Post-training alignment methods may induce sycophancy that deteriorates GenAI models' ability to disagree and push back — related
• Removing unsupervised pre-training hurts performance across all tasks, causing a 14.8% decrease relative to the full model — related