Single-turn tutors disclose answers pervasively while challenge scores are near zero across every model and subject¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Answer disclosure is pervasive in single-turn responses, particularly in Chemistry where Qwen2.5-72B gives away answers in 37.9% of responses. → Rima Hazra 2026 q2 i? Challenge scores are near zero across every model and subject, with the highest observed value being 0.21 for Llama-3.1-70B in Mathematics. → Rima Hazra 2026 (2)

Evidence¶

Rima Hazra 2026¶

Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, Mykola Pechenizkiy. (2026). SAFETUTORS: Benchmarking Pedagogical Safety in AI Tutoring Systems. arXiv preprint. https://arxiv.org/abs/2603.17373 doi:10.48550/arxiv.2603.17373

q2 · i? · design · r2

Single-turn process-level pedagogical analysis (Figure 3) across subjects and models, evaluating answer disclosure, challenge, topic maintenance, and clarity. The article reports "answer disclosure is pervasive", with Qwen2.5-72B giving away answers in 37.9% of Chemistry responses; no effect size is printed.

"However, answer disclosure is pervasive, particu- larly in Chemistry where Qwen2.5-72B gives away answers in 37.9% of responses."

Rima Hazra 2026 (2)¶

Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, Mykola Pechenizkiy. (2026). SAFETUTORS: Benchmarking Pedagogical Safety in AI Tutoring Systems. arXiv preprint. https://arxiv.org/abs/2603.17373

q2 · i? · design · r2

Single-turn pedagogical quality analysis across all evaluated models and subjects. The article reports "challenge scores are near zero across every model and subject", with the highest value 0.21 for Llama-3.1-70B in Mathematics, indicating models default to passive explanation; no effect size is printed.

"More critically, challenge scores are near zero across ev- ery model and subject, with the highest observed value being just 0.21 for Llama-3.1-70B in Math- ematics."

Discussion¶

Related Claims¶

• Model scale does not consistently improve pedagogical safety: within the Qwen2.5 family the 72B model beats the 7B on only 17 of 33 subject-dimension pairs — related
• No evaluated LLM tutor is reliably safe: every model exceeds 60% harm rate on at least five risk categories in single-turn and six in multi-turn evaluation — related
• Answer-revealing tutor turns are followed by less independent student work on every tutor base, a deterministic judge-invariant result — related
• Preference explanations favoring LearnLM more often cite staying on topic, challenging the learner, and not giving away answers — related
• Model scale alone does not induce Socratic behavior: unaligned Qwen-72B achieves 0% scaffolding effectiveness and 96.7% keyword leakage — related
• Multi-turn interaction amplifies pedagogical harm: mean harm rises 6-11 percentage points across subjects and pedagogical-relationship harm surges from 17.7% to 77.8% — related
• For revealing_answer, pedagogically undesired feedback achieves higher SuccScore in both semesters, suggesting success via copying revealed answers — related
• AI use is predominantly independent of the science subject taught, with one exception: chemistry teachers most often cite lack of subject-specific relevance — related
• Students recognized the harm of answer-revealing images only when images clearly revealed the answer — related