Newer LLM versions generally outperformed older versions on dental examination questions¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across multiple included studies, newer model versions achieved higher accuracy than their predecessors on dental examination questions (e.g., ChatGPT-4 over ChatGPT-3.5). → Kong Mengjie 2025

Evidence¶

Kong Mengjie 2025¶

Kong Mengjie, Fok Elise Hoi Wan, Yiu Cynthia Kar Yung. (2025). A Scoping Review of Large Language Models in Dental Education: Applications, Challenges, and Prospects. International Dental Journal. https://doi.org/10.1016/j.identj.2025.103854

q2 · i? · review · r2

The review reports, citing Danesh et al, that in a study evaluating ChatGPT-3.5 and ChatGPT-4 on 311 MCQs from an AAP in-service examination, "General accuracy for GPT-4 was significantly higher than GPT-3.5."

"General accuracy for GPT-4 was significantly higher than GPT-3.5. LLMs showed best performance on pharmacology and therapeutics."

Discussion¶

Related Claims¶

• LLMs have been applied across seven distinct domains in dental education, with dental examinations the most frequent (n = 24) — related
• Direct comparisons confirm ChatGPT-4 significantly outperformed earlier versions and some competitor models on dental examination questions — possibly the same claim (merge candidate)
• Newer OpenAI LLM versions achieve significantly higher pooled accuracy on biomedical MCQs than earlier versions (GPT‐4o–o1: 90%; GPT‐4: 82%; GPT‐3/3.5: 59%) — a broader claim this one bears on
• GPT‐4 through o1 models achieve pooled accuracy of 87% in the anatomical sciences and ≥83% in five of six biomedical disciplines, consistently above older versions — a broader claim this one bears on
• GPT-4 achieves passing accuracy on USMLE Steps 1–3 (mean 86%), significantly exceeding GPT-3.5 (mean 56%) — a broader claim this one bears on
• LLM version progression explains about half of the variance in biomedical MCQ accuracy, with GPT‐4o_o1 over six times as likely as GPT‐3/3.5 to answer correctly — a broader claim this one bears on
• Subgroup analyses show significant between-model differences, with Copilot (75.25%) and ChatGPT-4 (72.97%) achieving the highest pooled accuracies — related
• Large language models achieve a pooled accuracy of 63.69% (95% CI 60.26%-67.06%) in answering dental examination questions — related
• GPT-3.5 achieves an integrated accuracy of 54% on dental licensing examinations — related
• GPT-4 outperforms GPT-3.5 and Bard on dental licensing examinations, achieving an integrated accuracy of 72% — a narrower finding that bears on this claim