DAPO was selected over GRPO for the RL stage because its decoupled policy update gave more stable gradients and superior CDPK performance¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In ablation studies, DAPO's decoupling of the policy update from advantage estimation provided more stable gradients on multi-step pedagogical reasoning chains and yielded superior CDPK performance compared with GRPO. → Singh 2026

Evidence¶

Singh 2026¶

Singh, N. P., Wang, X., Garikipati, A., Ciobanu, M., Mao, Q., & Das, R. (2026). Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised Fine-Tuning. Preprint. https://arxiv.org/abs/2604.06385 doi:10.48550/arxiv.2604.06385

q2 · i? · design · r2

Ablation study comparing Group Relative Policy Optimization (GRPO) and Decoupled Advantage Policy Optimization (DAPO) during the first RL stage; the authors report DAPO yielded "superior performance on the CDPK Benchmark". No effect size or accuracy for the ablation is printed.

"We found that DAPO's ability to decouple the policy update from the advantage estimation provided more stable gradients when training on the complex, multi-step reasoning chains required for pedagogical scaffolding, ultimately yielding superior performance on the CDPK Benchmark, and was therefore selected as our primary RL algorithm."

Discussion¶

Related Claims¶

• Both RL stages are needed: removing DPO and GRPO training causes the largest performance drop in student simulation — related
• GRPO training shifts profiles from broad summaries toward diagnostic, behaviorally grounded representations that better support simulation — related