In the Opus-only ablation, a prompt-only Socratic instruction outscores the full routed policy on judged pedagogy at near-tied helpfulness¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The Socratic prompt-only variant outscores the full routed policy on judged pedagogy (paired difference +0.565, δ=+0.96, ten of ten replicate pairs). → Shuyi Fan 2026

Evidence¶

Shuyi Fan 2026¶

Shuyi Fan, Boyuan Deng, Mengyu Xu, Jiale Liu, Hongyang Zhang, Qiaoxin Yang, Chongyang Gao. (2026). Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models. arXiv. https://arxiv.org/abs/2607.28128 doi:10.48550/arxiv.2607.28128

q2 · i? · causal · r2

Descriptive Opus-only ablation of five registered variants against their baselines, outside the confirmatory tests; judged helpfulness stayed within a 0.25-point band across all seven policies while judged pedagogy spanned 2.3 points.

"the Socratic prompt-only variant outscores the full routed policy on judged pedagogy (paired difference+0.565, CI[+0.348,+0.817],δ=+0.96, ten of ten replicate pairs) at near-tiedhelpfulness."

Discussion¶

Related Claims¶

• On the primary base under the frozen Opus judge, ConvTutor and PedTutor do not differ significantly in judged helpfulness but are perfectly rank-separated in judged pedagogy — related
• The ConvTutor–PedTutor helpfulness ordering is judge-contingent, reversing between the Opus and Sol judges on two of three tutor bases — related
• On the primary base, a turn that leaks the answer predicts a higher Opus helpfulness rating and less next-turn independent student work — related