Expert raters rate TeachingCoach higher than a GPT-4o baseline on clarity, tone, reflection, and acknowledgment of user input¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? TeachingCoach consistently outperformed GPT-4o on expert ratings of clarity, respectful tone, encouragement of reflection, and acknowledgment of user input over 200 conversations. → Isabel Molnar 2026

Evidence¶

Isabel Molnar 2026¶

Isabel Molnar, Peiyu Li, Si Chen, Sugana Chawla, James Lang, Ronald Metoyer, Ting Hua, Nitesh V. Chawla. (2026). TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors. https://arxiv.org/abs/2603.18189 doi:10.48550/arxiv.2603.18189

q2 · i? · design · r2

Expert evaluation of 200 conversations on a 3-point scale across four dimensions. Table 1 reports TeachingCoach scoring 2.62, 2.55, 2.59, 2.56 on E1-E4 versus GPT-4o baseline at 2.01, 1.99, 1.68, 1.85; the article states "TeachingCoach consistently outperformed GPT-4o."

"We asked teaching experts to evaluate 200 conversations generated byTeachingCoachand a GPT-4o baseline. Each dialogue turn was rated on four dimensions:E1 clarity of responses,E2 respectful tone,E3 encouragement of reflection and reasoning, andE4 acknowledg- ment of user input."

Discussion¶

Related Claims¶

• Users experienced a trade-off between conversational depth and interaction efficiency between the fine-tuned and baseline chatbots — related
• In a user study, the GPT-4o-mini baseline was preferred overall, but participants more often attributed learning to the fine-tuned TeachingCoach model — related
• Making trustworthiness metrics and visualizations explicit increased inter-rater reliability among learning engineers evaluating LLM responses — related
• LLM ratings of classroom transcripts are more correlated with each other than with expert human ratings, across the same and different tasks — related
• LLM feedback receives higher expert ratings on six of seven quality dimensions, with tone the exception — related
• The proposed framework outperforms fine-tuned BERT, Instructor-LM, GPT-4, and RAPT on TeacherEval-2023 across consistency, credibility, calibration, and fairness metrics — related