GRPO alignment without the directness reward raises Socratic scaffolding effectiveness from 0.300 to 0.633 and cuts keyword leakage from 26.7% to 13.3% on held-out questions¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · checked against its sources 2026-10-17 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? On 30 held-out questions, PT-GRPO w/o Rdir reaches SE=0.633 with 13.3% leakage, up from PT-Base's SE=0.300 with 30.0% leakage. → Xiaokun Wang 2026

Evidence¶

Xiaokun Wang 2026¶

Xiaokun Wang, Siyu Song, Wentao Liu, Xiaodong Zou. (2026). Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning. arXiv preprint. https://arxiv.org/abs/2607.22996 doi:10.48550/arxiv.2607.22996

q2 · i? · causal · r1

Simulation-based evaluation on 30 held-out seed questions in a four-turn LLM-simulated tutoring loop, comparing PT-track systems. The article reports "GRPO without Rdir further raises SE to 0.633" with leakage cut to 13.3%; per-run SE variance is stated as roughly ±0.10.

"Within the PT track, alignment yields a generally improving pattern: prompt-only PT-Base reaches SE=0.300; SFT alone improves to 0.500; GRPO without Rdir further raises SE to 0.633 while cutting leakage from 26.7% to 13.3%."

Discussion¶

Related Claims¶

• Using the directness penalty Rdir as a GRPO reward is counterproductive: it lowers SE by 0.200 and raises leakage by 10 percentage points relative to omitting it — related
• Gradient-based GRPO alignment of a 7B model outperforms prompt engineering on scaffolding effectiveness (0.633 vs 0.400) despite the smaller backbone — related
• The best GRPO variant also achieves the highest Conversation Depth (0.722), suggesting stronger scaffolding accompanies deeper semantic progression — related
• Supervised fine-tuning on SocraticEdu provides the largest single-step SE gain (+0.200) but still leaks target terms in 26.7% of dialogues — related
• GRPO alignment collapses Mathematical-Knowledge-for-Teaching utterances and surges Socratic-style Exploratory questions in tutor output — related
• Model scale alone does not induce Socratic behavior: unaligned Qwen-72B achieves 0% scaffolding effectiveness and 96.7% keyword leakage — related