MisconceptionTutor's advantage in feedback success (SuccScore) is mixed: substantial on earlier assignments, diminishing or reversing on later ones¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? MisconceptionTutor substantially improves SuccScore on earlier assignments, but differences diminish or reverse on later assignments, with no significant improvement on Assignments 3 and 5. → Rose Niousha 2026

Evidence¶

Rose Niousha 2026¶

Rose Niousha, Samantha Boatright Smith, Bita Akram, Peter Brusilovsky, Arto Hellas, Juho Leinonen, John DeNero, and Narges Norouzi. (2026). The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness. arXiv preprint. https://arxiv.org/abs/2605.05648 doi:10.48550/arxiv.2605.05648

q2 · i? · associational · r2

Per-assignment SuccScore comparison (Table 3): significant differences on Assignments 1, 2, and 4 (Assignment 4 favoring BaselineTutor, p = 0.035), and non-significant differences on Assignments 3 (p = 0.469) and 5 (p = 0.459).

"MisconceptionTutorsubstantially improves success on earlier assignments, but differences diminish or reverse on later assignments, with no significant improvement on Assignments 3 and 5."

Discussion¶

Related Claims¶

• MisconceptionTutor achieves higher pedagogical quality (DAMR) than BaselineTutor on nearly all dimensions, with small-to-moderate effect sizes — related
• MisconceptionTutor achieves substantially higher feedback relevance (RelScore) than BaselineTutor on every assignment — related
• Pedagogically desired feedback is engaged with more (higher RelScore), but pedagogical quality shows little relation to whether actions are applied correctly (SuccScore) — related
• Engagement-based metrics are more robust predictors of students' perceived feedback helpfulness than pedagogical dimensions — related