GPT‐4 through o1 models achieve pooled accuracy of 87% in the anatomical sciences and ≥83% in five of six biomedical disciplines, consistently above older versions¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? GPT‐4_o1 versions achieved 87% pooled accuracy for the anatomical sciences versus 83% for GPT‐3/3.5, and ≥83% for USMLE Step 1/Equivalent and four other disciplines. → Cheverko CM 2026

Evidence¶

Cheverko CM 2026¶

Cheverko CM, Mavrych V, Bolgova O, Mohamed FRR, Westrick J, Juarez L, Rush E, Solka KA, Doubleday AF, Byram JN, Becker R, Gomez V, Ganeng BKA, Hoffman LA, Roach VA, Brown KM, DeVaul N, Garnett CN, Herriott HL, Lufler RS, Mussell JC, Balta JY, Pascoe MA, Middleton JW, Duffy S, Stephens GC, Wilson AB. (2026). The performance of ChatGPT and other large language models on multiple‐choice questions in biomedical disciplines: A meta‐analysis. Anatomical Sciences Education. https://doi.org/10.1002/ase.70262

q3 · i? · quant-synthesis · r2

Discipline‐level pooled analysis including disciplines testing OpenAI LLMs in ≥5 studies, comparing GPT‐4 through o1 (GPT‐4_o1) with GPT‐3/3.5. Reports "a pooled accuracy of 87% for the anatomical sciences, compared with 83% for GPT‐3/3.5"; no effect size printed. Figure 3 shows pooled accuracy by discipline and version.

"GPT‐4_o1 versions achieved a pooled accuracy of 87% for the anatomical sciences, compared with 83% for GPT‐3/3.5 (Figure 2). GPT‐4_o1 demonstrated a pooled accuracy of ≥83% for “USMLE Step 1/Equivalent” and four other biomedical disciplines (Figure 3)."

Discussion¶

Related Claims¶

• GPT‐4 through o1 models show significantly lower pooled accuracy in pathology (75%) than in other biomedical disciplines — related
• Newer OpenAI LLM versions achieve significantly higher pooled accuracy on biomedical MCQs than earlier versions (GPT‐4o–o1: 90%; GPT‐4: 82%; GPT‐3/3.5: 59%) — related
• LLM version progression explains about half of the variance in biomedical MCQ accuracy, with GPT‐4o_o1 over six times as likely as GPT‐3/3.5 to answer correctly — related
• Newer Anthropic and DeepSeek models approach GPT‐4o–o1 accuracy on biomedical MCQs, while Google Gemini and Microsoft Copilot trail the leading companies — related
• GPT-4 achieves passing accuracy on USMLE Steps 1–3 (mean 86%), significantly exceeding GPT-3.5 (mean 56%) — related
• Question type, stem type, MCQ source, and web accessibility did not independently explain LLM accuracy heterogeneity in biomedical MCQ meta‐regressions — related
• Newer LLM versions generally outperformed older versions on dental examination questions — a narrower finding that bears on this claim