MisconceptionTutor achieves substantially higher feedback relevance (RelScore) than BaselineTutor on every assignment¶
Claim · All claims
Evidence · 1 study · associationalr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Across all five assignments, MisconceptionTutor's feedback relevance exceeds BaselineTutor's by 9–21 percentage points, with statistically significant differences on every assignment. → Rose Niousha 2026
Evidence¶
Rose Niousha 2026¶
Rose Niousha, Samantha Boatright Smith, Bita Akram, Peter Brusilovsky, Arto Hellas, Juho Leinonen, John DeNero, and Narges Norouzi. (2026). The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness. arXiv preprint. https://arxiv.org/abs/2605.05648 doi:10.48550/arxiv.2605.05648
q2 · i? · associational · r2
Per-assignment engagement comparison of the two tutors (Table 3) using Mann–Whitney U tests with Holm-corrected p-values; the quote reports "improvingrelevance by 9–21 percentage points on every assignment" with significance on all five.
"Across all assignments,MisconceptionTutorachievessubstantiallyhigherRelScore,improvingrelevance by 9–21 percentage points on every assignment."
Discussion¶
Related Claims¶
- MisconceptionTutor's advantage in feedback success (SuccScore) is mixed: substantial on earlier assignments, diminishing or reversing on later ones — related
- Engagement-based metrics are more robust predictors of students' perceived feedback helpfulness than pedagogical dimensions — related
- Pedagogically desired feedback is engaged with more (higher RelScore), but pedagogical quality shows little relation to whether actions are applied correctly (SuccScore) — related
- MisconceptionTutor achieves higher pedagogical quality (DAMR) than BaselineTutor on nearly all dimensions, with small-to-moderate effect sizes — related
- For revealing_answer, pedagogically undesired feedback achieves higher SuccScore in both semesters, suggesting success via copying revealed answers — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
misconception-tutor-higher-relscore-all-assignments |
title |
MisconceptionTutor achieves substantially higher feedback relevance (RelScore) than BaselineTutor on every assignment |
description |
MisconceptionTutor achieves substantially higher feedback relevance (RelScore) than BaselineTutor on every assignment |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — rose-niousha-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.