In in-training residency exams, ChatGPT-4.0 achieved 72.2% accuracy compared to 57.7% for ChatGPT-3.5¶
Claim · All claims
Evidence · 1 study · quant-synthesisr2·q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? In in-training residency exams, ChatGPT-4.0 achieved an accuracy rate of 72.2% compared to 57.7% for ChatGPT-3.5. → Lesselroth Blake 2025
Evidence¶
Lesselroth Blake 2025¶
Lesselroth Blake, Riese Janik, Yang Zhichao, Jaleel Shahila, Shabbir Waqas, Jaleel Anila, Aziz Umair, Farid Ghulam, Zahid Bashir Muhammad, Mirza Tehmasp Rehman, Khizar Abbas Syed Mohammad, Aslam Shiraz, Sikander Rana Muhammad Hassaan. (2025). Evaluating the Potential and Accuracy of ChatGPT-3.5 and 4.0 in Medical Licensing and In-Training Examinations: Systematic Review and Meta-Analysis. JMIR Medical Education. https://doi.org/10.2196/68070
q3 · i? · quant-synthesis · r2
Pooled analysis of in-training residency exam studies found ChatGPT-4.0 "achieved an accuracy rate of 72.2%, compared to 57.7% for ChatGPT-3.5". No standardized effect size is printed for this contrast, so impact is null.
"In in-training residency exams, ChatGPT-4 achieved an accuracy rate of 72.2%, compared to 57.7% for ChatGPT-3.5."
Discussion¶
Related Claims¶
- ChatGPT-4.0 outperforms ChatGPT-3.5 in medical licensing exams, achieving a pooled accuracy of 81.8% versus 60.8% — a broader claim this one bears on
- Subgroup analyses show significant between-model differences, with Copilot (75.25%) and ChatGPT-4 (72.97%) achieving the highest pooled accuracies — related
- ChatGPT-4.0 was 36% more likely than ChatGPT-3.5 to provide correct answers across medical licensing and residency exams — related
- Pooled LLM accuracy differs minimally by question source (71% for internal exams vs. 73% for public databases) — related
- Large language models achieve a pooled accuracy of 72% on medical licensing examination questions across 36 studies — related
Page metadata
| Field | Value |
|---|---|
status |
review |
generated |
by process:wiki-ingest on 2026-10-31 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.