Strict knowledge acquisition shows a smaller positive pooled signal for LLM-based teaching (SMD 0.80)¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q3 · i3 large
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i3 A sensitivity analysis restricted to stricter post-course knowledge outcomes (5 studies; 164 vs. 164 participants) yielded a pooled SMD 0.80 (95% CI 0.07 to 1.53; I² = 88.6%). → Liu Xiaoliu 2026

Evidence¶

Liu Xiaoliu 2026¶

Liu Xiaoliu, Pan Jiayue, Zhang Chengyi, Wang Xiaomei, Wan Jinghua, Wang Qiang. (2026). Effectiveness of large language model-based teaching interventions in medical education: a systematic review and stratified meta-analysis of 14 studies. BMC Medical Education. https://doi.org/10.1186/s12909-026-09879-0

q3 · i3 · quant-synthesis · r2

Domain-specific supplementary random-effects meta-analysis of 5 studies restricted to outcomes judged to represent stricter post-course knowledge acquisition. The printed effect size was SMD 0.80 (95% CI 0.07 to 1.53, I² = 88.6%); GRADE-informed certainty was low.

"The strict knowledge acquisition sensitivity analysis included 5 studies (164 vs. 164 participants; Fig. 4) and yielded a pooled effect of SMD 0.80 (95% CI 0.07 to 1.53; I² = 88.6%)."

Discussion¶

Related Claims¶

• Delayed retention shows no clear difference between LLM-based and traditional teaching (SMD 0.15) — related
• LLM-based teaching is associated with higher post-course score-based outcomes than non-AI teaching in an exploratory pooled synthesis (SMD 0.90), with very high heterogeneity — related
• Exploratory subgroup analysis suggests larger effects in practice courses than theory courses, driven by only 2 practice-oriented studies — related
• Structured clinical performance shows the largest pooled signal (SMD 3.39) but rests on a very small evidence base — related