SAFETUTORS benchmark

Product or Programme · All products and programmes
Evidence · 8 claims (8 for) · 2 studies (2 design), q2 · 0 of 2 report an effect size · 8 claims rest on one study

Description

A benchmark dataset of single-turn and multi-turn mathematics, physics, and chemistry tutoring dialogues, created and human-validated by Rima Hazra et al. for joint safety and pedagogical evaluation of AI tutors.

Components

Claims

• Informational harm is the sole dimension that decreases in multi-turn interaction, dropping from 60.52% to 11.90% [+W]
• Harm profiles are subject-dependent: metacognitive harm crosses 90% in all three subjects in multi-turn, peaking at 97.44% for Qwen2.5-32B in Mathematics [+W]
• Extended multi-turn dialogue does not produce effective tutoring: learning gains remain uniformly low and misconception reduction peaks around 24-25% [+W]
• Multi-turn interaction amplifies pedagogical harm: mean harm rises 6-11 percentage points across subjects and pedagogical-relationship harm surges from 17.7% to 77.8% [+W]
• No evaluated LLM tutor is reliably safe: every model exceeds 60% harm rate on at least five risk categories in single-turn and six in multi-turn evaluation [+W]
• Single-turn tutors disclose answers pervasively while challenge scores are near zero across every model and subject [+W]
• Model scale does not consistently improve pedagogical safety: within the Qwen2.5 family the 72B model beats the 7B on only 17 of 33 subject-dimension pairs [+W]
• LLM attack success rates increase consistently from single-turn to static multi-turn and dynamic multi-turn interactions in educational settings [+W]

Related Products and Programmes

-

Key Sources

• Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, Mykola Pechenizkiy. (2026). SAFETUTORS: Benchmarking Pedagogical Safety in AI Tutoring Systems. arXiv preprint. https://arxiv.org/abs/2603.17373

Page metadata

Field	Value
type	product
id	safetutors-benchmark
title	SAFETUTORS benchmark
description	A benchmark dataset of single-turn and multi-turn mathematics, physics, and chemistry tutoring dialogues, created and human-validated by Rima Hazra et al.
status	review
product_kind	dataset
generated	by process:settle-candidates on 2026-10-11
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.