After excluding the Dhok et al. (2023) outlier, students score significantly lower on VSAQs than MCQs¶

Claim · All claims
Evidence · 1 study · quant-synthesis r1 · q3 · i2 medium
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 Excluding the outlier cohort, the pooled SMD of -0.86 (95% CI [-1.01, -0.70]) shows students scored significantly lower on VSAQs than MCQs, with heterogeneity reduced from 98% to 4%. → Sher Ali 2026

Evidence¶

Sher Ali 2026¶

Sher Ali, Arshad Muhammad Moin Ud Din, Abbas Mesum, Asghar Abdul Hannan, Khan Anees Hussain, Ali Junaid, Yaqoob Muhammad Ibrahim, Alam Rana Aftab, Akram Muhammad Bilal, Masood Muhammad Abdullah, Thakur Amit Kumar. (2026). Very short answer questions vs. multiple choice questions in health professions education: a systematic review and meta-analysis. BMC Medical Education. https://doi.org/10.1186/s12909-026-09359-5

q3 · i2 · quant-synthesis · r1

Pre-specified sensitivity analysis of the score meta-analysis removing the Dhok et al. (2023) outlier. The pooled SMD of -0.86 favored MCQs, and "The removal of the outlier successfully reduced heterogeneity from 98% to 4%".

"After excluding the outlier (Dhok et al.), the sensitivity analysis (n = 5 cohorts) demonstrated a statistically significant difference favoring MCQs. The pooled Standardized Mean Difference was − 0.86 (95% CI [-1.01, -0.70]; p < 0.00001), indicating that students scored significantly lower on VSAQs compared to MCQs."

Discussion¶

Related Claims¶

• Initial pooled analysis shows no statistically significant difference between VSAQ and MCQ scores but with extreme heterogeneity — related
• VSAQ assessments demonstrate strong discriminatory power between high- and low-performing students — related
• Clinical analysis skills effect size is unstable and sensitive to two studies, dropping to d = 0.50 after their exclusion — related