No significant difference in accuracy between ChatGPT 3.5 and Bard 1.0 generated MCQs¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Pairwise analysis demonstrated no significant difference in accuracy between ChatGPT 3.5 and Bard 1.0. → Riehm Lauren 2026

Evidence¶

Riehm Lauren 2026¶

Riehm Lauren, Nanji Keean, Lakhani Moiz, Pankiv Evelina, Hasanee Dean, Pfeifer Wesla. (2026). The use of large language models in generating multiple choice questions for health professions education: A systematic review and network meta-analysis. PLOS One. https://doi.org/10.1371/journal.pone.0340277

q2 · i? · quant-synthesis · r2

Pairwise meta-analysis of studies comparing ChatGPT 3.5 and Bard 1.0 found "no significant difference in accuracy between Chat GPT3.5 and Bard". No human control arm was available for this outcome.

"Pairwise analysis demonstrated that there was no significant difference in accuracy between Chat GPT3.5 and Bard (RR: 0.84; 95%CI: 0.59 to 1.22; GRADE: Very Low)"

Discussion¶

Related Claims¶

• GPT-4 achieves the highest pooled accuracy among LLMs (81%), significantly outperforming GPT-3.5 (60%) — related
• ChatGPT-4.0 outperformed ChatGPT-3.5 and Bard on the AAMC SJT in both raw and adjusted accuracy — related
• No significant difference in item discrimination index or point biserial correlation between ChatGPT 3.5 and human-generated MCQs — related
• Chat-GPT generated more valid physiology MCQs than Bard and Bing, but at a lower difficulty level — reports the opposite