Users experienced a trade-off between conversational depth and interaction efficiency between the fine-tuned and baseline chatbots¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The fine-tuned model supported learning through sustained engagement and reflective questioning but required more turns, while the baseline prioritized speed and breadth of suggestions. → Isabel Molnar 2026

Evidence¶

Isabel Molnar 2026¶

Isabel Molnar, Peiyu Li, Si Chen, Sugana Chawla, James Lang, Ronald Metoyer, Ting Hua, Nitesh V. Chawla. (2026). TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors. https://arxiv.org/abs/2603.18189 doi:10.48550/arxiv.2603.18189

q2 · i? · design · r2

Qualitative findings from the 41-instructor user study, drawn from surveys and 15-minute post-study interviews. Participants distinguished the models on conversational engagement, efficiency of interaction, and breadth of suggestions; "Overall the findings highlight a trade-off between depth and efficiency."

"Overall the findings highlight a trade-off between depth and effi- ciency: the fine-tuned model supported learning through sustained engagement and reflection, while the baseline model prioritized speed and breadth of suggestions."

Discussion¶

Related Claims¶

• In a user study, the GPT-4o-mini baseline was preferred overall, but participants more often attributed learning to the fine-tuned TeachingCoach model — related
• Expert raters rate TeachingCoach higher than a GPT-4o baseline on clarity, tone, reflection, and acknowledgment of user input — related
• Fine-tuning frontier LLMs on authentic tutoring data improves performance on all six conversational and pedagogical benchmarks, moving values toward human-tutor levels — related
• The fine-tuned chatbot LlamaLo extensively expanded the user's dialogic space through deepening and widening acts across ten contentious-topic discussions — related
• Agreement between chatbot preference and perceived learning attribution was low (39%) — related
• How a user engages with AI matters more than whether: structured, deliberate engagement can mitigate or reverse offloading effects — a broader claim this one bears on