GPT-4 achieves passing accuracy on USMLE Steps 1–3 (mean 86%), significantly exceeding GPT-3.5 (mean 56%)¶

Claim · All claims
Evidence · 1 study · review r3 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? Across 9 USMLE benchmark studies, GPT-4 achieved mean accuracy of 86% (IQR 81–89%), above the ~60% passing standard, while GPT-3.5 averaged 56% (IQR 52–62%). → Bodke Parag 2026

Evidence¶

Bodke Parag 2026¶

Bodke Parag, Jumle Rahul. (2026). Large language models in undergraduate medical education: a systematic review of examination performance, knowledge acquisition, and learner perceptions. BMC Medical Education. https://doi.org/10.1186/s12909-026-09999-7

q3 · i? · review · r3

Narrative synthesis of 9 USMLE benchmark evaluation studies within the review's Results. The review reports "GPT-4 achieved a mean accuracy of 86%" versus GPT-3.5 at 56%; no pooled effect size was computed due to heterogeneity.

"Across 9 USMLE benchmark studies, GPT-4 achieved a mean accuracy of 86% (IQR: 81%–89%), substantially above the historically defined passing standard of approximately 60%. GPT-3.5 mean accuracy was 56% (IQR: 52%–62%)"

Discussion¶

Related Claims¶

• GPT‐4 through o1 models achieve pooled accuracy of 87% in the anatomical sciences and ≥83% in five of six biomedical disciplines, consistently above older versions — related
• GPT-4 exceeds pass marks on India NEET-PG and UK PLAB 1 but falls below pass on clinical reasoning and procedural items — related
• Epi2Diff achieves the best performance across all four item-difficulty test sets and all reported metrics, outperforming SLM fine-tuning, LLM prompting, and supervised LLM adaptation baselines — related
• GPT-4 examination accuracy is moderated by item type: multi-step vignettes (61–71%) and image-dependent items (48–63%) fall well below single-fact recall (83–88%) — related
• Newer Anthropic and DeepSeek models approach GPT‐4o–o1 accuracy on biomedical MCQs, while Google Gemini and Microsoft Copilot trail the leading companies — related
• LLM version progression explains about half of the variance in biomedical MCQ accuracy, with GPT‐4o_o1 over six times as likely as GPT‐3/3.5 to answer correctly — related
• Newer OpenAI LLM versions achieve significantly higher pooled accuracy on biomedical MCQs than earlier versions (GPT‐4o–o1: 90%; GPT‐4: 82%; GPT‐3/3.5: 59%) — related
• GPT-4 achieves the highest pooled accuracy among LLMs (81%), significantly outperforming GPT-3.5 (60%) — related
• LLMs, particularly GPT-4, can match or exceed medical students' examination performance — a broader claim this one bears on
• Studies using both text- and image-based questions show higher pooled LLM accuracy (78%) than text-only studies (71%) — related
• ChatGPT-4.0 outperforms ChatGPT-3.5 in medical licensing exams, achieving a pooled accuracy of 81.8% versus 60.8% — a broader claim this one bears on
• Newer LLM versions generally outperformed older versions on dental examination questions — a narrower finding that bears on this claim
• LLMs perform worse on dental licensing examinations than on medical licensing examinations — related