LLM-based teaching is associated with higher post-course score-based outcomes than non-AI teaching in an exploratory pooled synthesis (SMD 0.90), with very high heterogeneity¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q3 · i3 large
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i3 Across 7 studies (248 vs. 251 participants), LLM-based teaching showed a pooled advantage in post-course score-based outcomes (SMD 0.90, 95% CI 0.18 to 1.62, I² = 92.5%). → Liu Xiaoliu 2026

Evidence¶

Liu Xiaoliu 2026¶

Liu Xiaoliu, Pan Jiayue, Zhang Chengyi, Wang Xiaomei, Wan Jinghua, Wang Qiang. (2026). Effectiveness of large language model-based teaching interventions in medical education: a systematic review and stratified meta-analysis of 14 studies. BMC Medical Education. https://doi.org/10.1186/s12909-026-09879-0

q3 · i3 · quant-synthesis · r2

Exploratory inverse-variance random-effects meta-analysis of 7 studies (248 intervention vs. 251 control participants) pooling related but non-identical score-based endpoints. The printed effect size is SMD 0.90 (95% CI 0.18 to 1.62, I² = 92.5%, P = 0.015); GRADE-informed certainty was very low for this synthesis.

"The pooled analysis suggested that LLM-based teaching was associated with higher post-course score-based outcomes than non-AI control teaching, with a pooled effect size of SMD 0.90 (95% CI 0.18 to 1.62; I² = 92.5%; P = 0.015)."

Discussion¶

Related Claims¶

• Delayed retention shows no clear difference between LLM-based and traditional teaching (SMD 0.15) — related
• Exploratory subgroup analysis suggests larger effects in practice courses than theory courses, driven by only 2 practice-oriented studies — a narrower finding that bears on this claim
• Strict knowledge acquisition shows a smaller positive pooled signal for LLM-based teaching (SMD 0.80) — related
• Structured clinical performance shows the largest pooled signal (SMD 3.39) but rests on a very small evidence base — related