HAL's text-based human-likeness advantage did not transfer to embodied audiovisual interaction, and measured 3E performance was virtually identical between models¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-17 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? In the embodied setting, 54% of participants judged GPT-4o-mini more natural and human-like versus 22% for HAL, while mean overall 3E scores were nearly identical (0.437 versus 0.438; p=0.481). → Masum Hasan 2026

Evidence¶

Masum Hasan 2026¶

Masum Hasan, Ron Epstein, Thomas Carroll, Ehsan Hoque. (2026). Scalable AI-based clinical communication training and automated assessment. https://arxiv.org/abs/2609.22517 doi:10.48550/arxiv.2609.22517

q2 · i? · causal · r2

Counterbalanced comparison of the two conversational backends within the 59-participant embodied study. "Measured 3E performance did not differ between the models", despite participants more often judging GPT-4o-mini (54%) more natural and human-like than HAL (22%).

"Measured 3E performance did not differ between the models: mean overall scores were 0.437 for GPT-4o-mini and 0.438 for HAL (difference, 0.001; 95% CI,−0.034–0.036; one-sidedp= 0.481)."

Discussion¶

Related Claims¶

• An automated LLM assessor rates 3E clinical communication skills with agreement comparable to individual human raters — related
• FSM-based evaluation discriminates interactive quality across models, with GPT-5 Mini scoring highest and most tiers significantly separated — related
• Participants with prior communication-training experience achieved higher overall 3E scores than those without — related
• Overall 3E, Empower, and Empathize scores were higher in the second AI-patient encounter, but the uncontrolled design precludes attributing this to training — related