Studies using both text- and image-based questions show higher pooled LLM accuracy (78%) than text-only studies (71%)¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q3 i? Studies incorporating both text- and image-based questions achieved higher pooled accuracy than text-only studies (78% vs. 71%, P=0.03). → Nouri Haniyeh 2025

Evidence¶

Nouri Haniyeh 2025¶

Nouri Haniyeh, Mahdavi Abdollah, Abedi Ali, Mohammadnia Alireza, Hamedan Mahnaz, Amanzadeh Masoud. (2025). Performance of large language models in medical licensing examinations: a systematic review and meta-analysis. Journal of Educational Evaluation for Health Professions. https://doi.org/10.3352/jeehp.2025.22.36

q3 · i? · quant-synthesis · r3

Subgroup analysis by question format across the included studies (text-only 47 subgroup entries, text-and-image 16). The review reports 78% versus 71% with P=0.03, and attributes the improvement to multimodal prompting, particularly in GPT-4.

"When stratified by question format, studies incorporating both text- and image-based questions achieved slightly higher pooled accuracy compared with text-only studies (78% vs. 71%, P=0.03)."

Discussion¶

Related Claims¶

• Large language models achieve a pooled accuracy of 72% on medical licensing examination questions across 36 studies — related
• Multimodal input consistently improves accuracy but not F1, model size correlates moderately with performance, and reasoning models show mixed results — possibly the same claim (merge candidate)
• LLM accuracy drops from 96% on text-only to 74% on multimodal physics problems, with significant differences emerging among models — reports the opposite
• Newer OpenAI LLM versions achieve significantly higher pooled accuracy on biomedical MCQs than earlier versions (GPT‐4o–o1: 90%; GPT‐4: 82%; GPT‐3/3.5: 59%) — related
• GPT-4 achieves passing accuracy on USMLE Steps 1–3 (mean 86%), significantly exceeding GPT-3.5 (mean 56%) — related
• LLMs underperformed on image-based dental examination questions compared with text-based questions — reports the opposite
• LLMs perform worse on dental licensing examinations than on medical licensing examinations — related