In a user study, the GPT-4o-mini baseline was preferred overall, but participants more often attributed learning to the fine-tuned TeachingCoach model¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Among 41 higher education instructors comparing two chatbot versions, the baseline was preferred overall (21 vs. 13) while the fine-tuned model was more often credited with learning (18 vs. 13). → Isabel Molnar 2026

Evidence¶

Isabel Molnar 2026¶

Isabel Molnar, Peiyu Li, Si Chen, Sugana Chawla, James Lang, Ronald Metoyer, Ting Hua, Nitesh V. Chawla. (2026). TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors. https://arxiv.org/abs/2603.18189 doi:10.48550/arxiv.2603.18189

q2 · i? · causal · r2

Remote user study in August 2025 with 41 higher education instructors (22 faculty, 19 TAs or instructional staff), each exploring the same teaching challenge with both systems for up to 10 minutes per counterbalanced condition. "While the baseline model was preferred overall (21 vs. 13), participants more often attributed learning to the fine-tuned model (18 vs. 13)."

"While the baseline model was preferred overall (21 vs. 13), participants more often attributed learning to the fine-tuned model (18 vs. 13)."

Discussion¶

Related Claims¶

• Users experienced a trade-off between conversational depth and interaction efficiency between the fine-tuned and baseline chatbots — related
• Agreement between chatbot preference and perceived learning attribution was low (39%) — possibly the same claim (merge candidate)
• Expert raters rate TeachingCoach higher than a GPT-4o baseline on clarity, tone, reflection, and acknowledgment of user input — related