No evidence of publication bias is found in the meta-analysis of LLM examination accuracy¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Funnel-plot inspection and Egger's regression test showed no evidence of small-study effects (P=0.75). → Nouri Haniyeh 2025

Evidence¶

Nouri Haniyeh 2025¶

Nouri Haniyeh, Mahdavi Abdollah, Abedi Ali, Mohammadnia Alireza, Hamedan Mahnaz, Amanzadeh Masoud. (2025). Performance of large language models in medical licensing examinations: a systematic review and meta-analysis. Journal of Educational Evaluation for Health Professions. https://doi.org/10.3352/jeehp.2025.22.36

q3 · i? · quant-synthesis · r3

Publication bias assessment of the 36 included studies using funnel-plot visualization and Egger's regression test. The review reports "no evidence of small-study effects" with P=0.75.

"Visual inspection revealed no substantial asymmetry, consistent with Egger’s test results showing no evidence of small-study effects, thereby supporting the absence of publication bias (P=0.75)."

Discussion¶

Related Claims¶

• Large language models achieve a pooled accuracy of 72% on medical licensing examination questions across 36 studies — related
• Egger's regression found no statistically significant evidence of publication bias among the 28 collected studies — related
• Publication-bias checks indicate the pooled effect is unlikely to result from publication bias — related
• ChatGPT-4.0 was 36% more likely than ChatGPT-3.5 to provide correct answers across medical licensing and residency exams — related
• No evidence of publication bias in the SDL meta-analysis — related