Agreement between chatbot preference and perceived learning attribution was low (39%)¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Agreement between which system participants preferred and which they credited with learning was 39%, indicating preference drivers do not always match perceived instructional value. → Isabel Molnar 2026

Evidence¶

Isabel Molnar 2026¶

Isabel Molnar, Peiyu Li, Si Chen, Sugana Chawla, James Lang, Ronald Metoyer, Ting Hua, Nitesh V. Chawla. (2026). TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors. https://arxiv.org/abs/2603.18189 doi:10.48550/arxiv.2603.18189

q2 · i? · associational · r1

Secondary analysis of the 41-instructor user study comparing preference and learning-attribution responses. The article reports "Agreement between preference and learning attribution was low (39%)" and interprets this as a mismatch between preference drivers and perceived instructional value.

"Agreement between preference and learning attribution was low (39%), indicating that features driving preference do not always match perceived instructional value."

Discussion¶

Related Claims¶

• In a user study, the GPT-4o-mini baseline was preferred overall, but participants more often attributed learning to the fine-tuned TeachingCoach model — possibly the same claim (merge candidate)
• LLM-annotated empathy dimensions correlate positively with intention to follow, self-efficacy, and chatbot usefulness despite participants not preferring empathetic chatbots — related
• Users experienced a trade-off between conversational depth and interaction efficiency between the fine-tuned and baseline chatbots — related