LLM-annotated EPITOME scores show the Non-Empathetic chatbot expresses significantly lower empathy on all three dimensions, while Standard and Empathetic chatbots are not significantly different¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? One-way ANOVA on Gemini-annotated EPITOME scores over 237 conversations found significant between-condition differences; post hoc tests show the Non-Empathetic chatbot significantly lower on all three dimensions and Standard vs. Empathetic differences not significant. → Li Siyan 2026

Evidence¶

Li Siyan 2026¶

Li Siyan, Kai-Hui Liang, Shopnil Shahriar, Yilin Ye, Shiyoh Goetsu, Wei-Wei Du, Masahiro Yoshida, Tsunayuki Ohwa, Xuhai Xu, and Zhou Yu. (2026). Invisible Impact of Empathy on Behavioral Change: Isolating the Effect of Empathy in Long-term Physical Activity Coaching Chatbot Interactions. https://arxiv.org/abs/2606.26641 doi:10.48550/arxiv.2606.26641

q2 · i? · causal · r1

LLM-based annotation of 237 collected conversations (76 Non-Empathetic, 80 Standard, 81 Empathetic) using Gemini-2.5-Pro with the EPITOME three-dimensional empathy framework, one-way ANOVA with Bonferroni-corrected post hoc t-tests. No standardized effect size is printed; the article notes the Empathetic chatbot's average interpretation level is higher though not significant post-adjustment.

"In particular, the Non-Empathetic chatbot scores significantly lower along all three empathy dimensions. However, the differences in means between the Standard and Empathetic chatbots are not sufficiently significant."

Discussion¶

Related Claims¶

• In a six-week within-subject study, the Empathetic PA coaching chatbot improves intention to follow and self-efficacy faster over time than the Non-Empathetic condition — related
• Starting interaction with the Empathetic chatbot is associated with the least attrition in the six-week study — related
• LLM-annotated empathy dimensions correlate positively with intention to follow, self-efficacy, and chatbot usefulness despite participants not preferring empathetic chatbots — related
• Prompt settings and temperature settings have significant main effects on correlations among the three LLM chatbots — related
• Average check-in and weekly ratings show no significant differences between chatbot conditions, with the Non-Empathetic chatbot often rated as more engaging and useful — related
• Interview-containing prompts increased response diversity for Claude and GPT, while Gemini diversified with demographic-only prompts — related
• Claude showed the highest alignment with human BREQ responses, and interview-containing prompts aligned better than baseline prompts — related
• Non-expert participants often fail to notice empathy differences between chatbot versions, so self-reported perceived empathy alone is an unreliable assessment method — related
• REMind did not significantly increase measured empathy, but baseline human-directed empathy correlated positively with robot-directed empathy — related