Multi-agent AI standardized patient training improved final OSCE-aligned examination scores compared with a structured non-LLM control condition¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3 · i3 large
Currency · newest study 2026 · checked against its sources 2026-10-16 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q3 i3 Students trained with the multi-agent AI standardized patient system achieved higher final examination scores than the structured control group, with a large standardized effect size. → Yang 2026 q3 i? In a linear model, multi-agent assignment was associated with a 16.4-percentage-point higher final examination score, and a higher weighted checklist score. → Yang 2026 (2)

Evidence¶

Yang 2026¶

Yang, L., Liu, H., Li, S., Jia, R., Xiao, Y., Chen, G., & Lu, L. (2026). Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training. https://arxiv.org/abs/2609.10939 doi:10.48550/arxiv.2609.10939

q3 · i3 · causal · r2

Two-arm randomized controlled study of medical students (MA n = 47, CT n = 48 analyzed). The Mann-Whitney U test on final exam scores showed "a statistically significant between-group difference(P = 5.51 × 10−5)", with means of 71.8% (MA) vs 55.6% (CT); the article reports a large Hedges' g = −0.81 favoring MA, sign reflecting coding direction.

"The mean final examination score was71.8%in the multi-agent group and55.6%in the control group, with corresponding medians of78.8%and60 .6%, respectively. A Mann-Whitney U test showed a statistically significant between-group difference(P = 5.51 × 10−5)"

Yang 2026 (2)¶

Yang, L., Liu, H., Li, S., Jia, R., Xiao, Y., Chen, G., & Lu, L. (2026). Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training. https://arxiv.org/abs/2609.10939. doi:10.48550/arxiv.2609.10939

q3 · i? · causal · r2

Linear-model analysis of the same randomized comparison confirmed the exam-score advantage: "a 16.4-percentage-point higher final examination score" (β = 16.4; 95% CI, 7.7 to 25.1; P = 3.30 × 10−4). Multi-agent learning was also associated with a higher weighted checklist score (β = 11.7; 95% CI, 0.2 to 23.1; P = 0.046).

"Assignment to the multi-agent group was associated with a 16.4-percentage-point higher final examination score compared with the control group(β = 16.4; 95% CI, 7.7 to 25.1; P = 3.30 × 10−4)"

Discussion¶

Related Claims¶

• The largest OSCE-domain improvement from multi-agent scaffolding was in communication performance — a narrower finding that bears on this claim
• The largest checklist-item gain from multi-agent scaffolding was in observable empathic expression, with additional gains in selected history-taking items — related
• Simulation Based Education Improves Outcomes — a broader claim this one bears on
• Simulation Based Education With Deliberate Practice Improves Clinical Outcomes — a broader claim this one bears on
• Multi-agent scaffolding showed no significant difference from the structured control in binary diagnostic accuracy or worksheet completion — related
• Access to a structured conversational AI tutor increased post-test content scores over a no-AI holdout with ambient consumer AI available — related
• Controlled GenAI virtual-patient and communication studies reported short-term skill gains, but positive findings were not universal — a broader claim this one bears on
• AI-powered simulations show short-term gains in communication confidence and structured knowledge in RCTs, with limited or inferior effects for tactile psychomotor skills — related
• MPathic-VR-trained students achieve significantly higher composite scores than computer-based learning students on a subsequent advanced communication OSCE, several days later — related
• CLT-aligned e-learning modules improved cognitive load profiles and OSCE performance in simulation-based medical training (review-attributed to Gutierrez et al., 2023) — related