Expert tutors' lesson-planning and practice-design leaderboards separated AI models, and expert-rated practice design correlated with fewer student answer disputes¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? In 2,028 pairwise rubric reviews by 51 expert tutors, leaderboards distinguished models, Anthropic models (particularly Opus) performed strongly, and expert practice-design preference correlated with fewer student answer disputes (ρ=0.71 and 0.77 in Quantitative and Combined). → Curtis Northcutt 2026

Evidence¶

Curtis Northcutt 2026¶

Curtis Northcutt, Inaara Hasmani, Kevin Feng, Trevor Khangi, Andreas Plesner, Jonas Mueller. (2026). StudentBench: AI and human tutoring yield equivalent GRE learning gains. https://arxiv.org/abs/2609.28470 doi:10.48550/arxiv.2609.28470

q2 · i? · causal · r2

Second study: 51 expert human tutors completed 2,028 pairwise reviews of AI lesson plans and practice problems based on 381 student pre-tests; correlations with student flag rates were "Spearmanρ=0.71 and0.77". Correlational, exploratory; flags record disputes, not verified errors.

"Interestingly, AI tutors scoring highly on practice problem design (Figure3B), i.e., AI tutors preferred by experts for practice-problem creation, also tended to receive fewer student answer disputes in Quantitative and Combined (Spearmanρ=0.71 and0.77;p=.035andp=.033; Appendix F.2)."

Discussion¶

Related Claims¶

• Human raters' preferences align with the LLM judge on TutorBench (Pearson r = 0.82 across metric-level win rates) — related
• An LLM-based AI Evaluator agrees with expert human raters on collaboration transcripts at a level similar to inter-expert agreement — related
• LLM accuracy on the CDPK pedagogy benchmark ranges from 28% to 89%, with closed-weight reasoning models dominating the top of the leaderboard — related
• Automated metrics and LLM judges can misalign with expert human preferences in non-verifiable educational evaluation — related
• LLM-based rubric scores agreed with expert ratings at the total-score level and on cognitive-process dimensions, but agreement was low for more subjective dimensions — related