GPT-4 examination accuracy is moderated by item type: multi-step vignettes (61–71%) and image-dependent items (48–63%) fall well below single-fact recall (83–88%)¶
Claim · All claims
Evidence · 1 study · reviewr3·q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? GPT-4 accuracy on multi-step integration vignettes (61–71%) and image-dependent questions (48–63%) is substantially lower than on single-fact recall and pattern recognition items (83–88%), and the model shows systematically overconfident, poorly calibrated uncertainty. → Bodke Parag 2026
Evidence¶
Bodke Parag 2026¶
Bodke Parag, Jumle Rahul. (2026). Large language models in undergraduate medical education: a systematic review of examination performance, knowledge acquisition, and learner perceptions. BMC Medical Education. https://doi.org/10.1186/s12909-026-09999-7
q3 · i? · review · r3
Narrative synthesis of performance moderators across 19 examination studies in the review's Results. The review reports the named accuracy ranges and notes a systematic overconfidence pattern in five studies, with high confidence assigned to incorrect responses.
"GPT-4 accuracy on single-fact recall and pattern recognition items (83%–88%) substantially exceeded its accuracy on multi-step integration vignettes requiring sequential clinical decision-making (61%–71%). Performance on image-dependent questions (radiology, histopathology, dermatology) was disproportionately low (48%–63%)"
Discussion¶
Related Claims¶
- GPT-4 exceeds pass marks on India NEET-PG and UK PLAB 1 but falls below pass on clinical reasoning and procedural items — related
- GPT-4 achieves passing accuracy on USMLE Steps 1–3 (mean 86%), significantly exceeding GPT-3.5 (mean 56%) — related
- LLMs, particularly GPT-4, can match or exceed medical students' examination performance — related
- In the combined review analysis, ChatGPT-3.5 failed eight of nine UK postgraduate exams while ChatGPT-4 passed nine of eleven — related
Page metadata
| Field | Value |
|---|---|
status |
review |
generated |
by process:wiki-ingest on 2026-10-31 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.