Learner self-reported improvement after AI-generated virtual patient training is sometimes discordant with observed or cohort-comparison performance¶

Claim · All claims
Evidence · 1 study · review r3 · q2
Currency · newest study 2026 · checked against its sources 2026-10-31 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? In three studies, learners trained with AI-generated virtual patients scored no better than pre-training observed performance or lower than concurrent or previous cohorts without such training, despite self-reported improvements. → Fang Eric Kin Meng 2026

Evidence¶

Fang Eric Kin Meng 2026¶

Fang Eric Kin Meng, Boon Yuru, Ho Jin Yang, Tey Natasha, Low Matthew Jian Wen. (2026). Artificial intelligence generated virtual patients in health professions education: a scoping review. BMC Medical Education. https://doi.org/10.1186/s12909-026-10028-w

q2 · i? · review · r3

Qualitative synthesis of Kirkpatrick level 2 findings reported in three included studies of AI-generated virtual patient training in HPE; the review reports "scored no better than pre-training observed performance or lower" against comparison cohorts despite positive self-reports.

"Conversely, in three studies, learners trained using AI generated virtual patients scored no better than pre-training observed performance or lower compared to a concurrent or previous cohort without such training despite self-reported improvements"

Discussion¶

Related Claims¶

• Controlled GenAI virtual-patient and communication studies reported short-term skill gains, but positive findings were not universal — related
• Overall 3E, Empower, and Empathize scores were higher in the second AI-patient encounter, but the uncontrolled design precludes attributing this to training — related
• AI-powered simulations show short-term gains in communication confidence and structured knowledge in RCTs, with limited or inferior effects for tactile psychomotor skills — related
• Student satisfaction with virtual patients is generally positive, with item-level comparisons dominating and some confidence exceptions — related
• AI-generated virtual patients in HPE are used mainly for teaching and learning communication skills, diagnostic reasoning and management reasoning, with assessment a minority use case — related
• Outcome evaluation of AI-generated virtual patients in HPE is concentrated at Kirkpatrick level 1, with no studies reporting level 3 or 4 outcomes — related
• Repeated exposure to AI-VPs is associated with improved perceived self-efficacy and more positive attitudes toward communication training — reports the opposite
• Evidence of transfer from AI-VP training to real clinical practice is scarce, and one cohort study found lower field assessment scores after AI-VP training — a narrower finding that bears on this claim
• Virtual patient-based learning produces greater learning performance, self-efficacy and communication skills than conventional learning in a quasi-experimental study — reports the opposite