Number of judges positively influences cut scores, while test length influences pass rates¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Each additional judge was associated with a 0.19-percentage point increase in cut scores (p = 0.003), while each additional test item was associated with a 0.05-percentage point increase in the pass rate (p = 0.001). → Kannan 2025

Evidence¶

Kannan 2025¶

Kannan, S., & Gowri, S. (2025). Angoff methods in standard setting in health professional education: a systematic review and meta-analysis. BMC Medical Education, 25, 1727. doi:10.1186/s12909-025-08300-6

q2 · i? · quant-synthesis · r2

Meta-regression analyses explored heterogeneity by number of judges and test length. The analysis found "each additional judge was associated with a 0.19-percentage point increase in cut scores (p = 0.003)" and "each additional test item was associated with a 0.05-percentage point increase in the pass rate (p = 0.001)".

"Meta-regression indicated that each additional judge was associated with a 0.19-percentage point increase in cut scores (p = 0.003) while each additional test item was associated with a 0.05-percentage point increase in the pass rate (p = 0.001)."

Discussion¶

Related Claims¶

• Contextual factors (learner level, discipline, format) moderate Angoff outcomes
• Angoff methods produce significantly higher pass rates than conventional fixed standard-setting methods
• Angoff (Yes/No) variant shows the highest variability in cut scores
• Mastery Angoff produces the highest cut scores and lowest pass rates among Angoff variants
• The Angoff method produced lower cut-off scores and higher pass rates than Cohen and Hofstee methods in dental written assessments, especially in early and middle years