Model scale alone does not induce Socratic behavior: unaligned Qwen-72B achieves 0% scaffolding effectiveness and 96.7% keyword leakage¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Qwen-72B-Direct without task-specific alignment reaches SE=0.000 and leakage of 96.7% on the 30-question test set. → Xiaokun Wang 2026

Evidence¶

Xiaokun Wang 2026¶

Xiaokun Wang, Siyu Song, Wentao Liu, Xiaodong Zou. (2026). Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning. arXiv preprint. https://arxiv.org/abs/2607.22996 doi:10.48550/arxiv.2607.22996

q2 · i? · causal · r1

Scale-baseline condition in the same 30-question simulated evaluation (Table 3). The article reports "Qwen-72B without task-specific alignment achieves 0%SE and 96.7% leakage", matching the direct-answerer failure mode.

"Qwen-72B without task-specific alignment achieves 0%SE and 96.7% leakage, matching the direct-answering failure observed in the unaligned API baseline rather than the prompted PT-Base system."

Discussion¶

Related Claims¶

• Model scale does not consistently improve pedagogical safety: within the Qwen2.5 family the 72B model beats the 7B on only 17 of 33 subject-dimension pairs — a broader claim this one bears on
• Single-turn tutors disclose answers pervasively while challenge scores are near zero across every model and subject — related
• Gradient-based GRPO alignment of a 7B model outperforms prompt engineering on scaffolding effectiveness (0.633 vs 0.400) despite the smaller backbone — related
• GRPO alignment without the directness reward raises Socratic scaffolding effectiveness from 0.300 to 0.633 and cuts keyword leakage from 26.7% to 13.3% on held-out questions — related
• Prompt specificity changes AI tutor pedagogy: a no-direct-answer guardrail alone produces uniform questioning, while a detailed guided protocol produces varied, impasse-sensitive responses — related