Percentage rating format in the Modified Angoff method yields higher reliability than the Yes/No format, with ICCs of 0.48–0.65 versus a maximum of 0.41¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Using percentage ratings in the Modified Angoff method produced higher reliability (ICC 0.48–0.65) than the Yes/No option (maximum ICC 0.41), and response format significantly impacted pass rates, which ranged from 40.3% to 89.6%. → Lin Galvin Sim Siang 2026

Evidence¶

Lin Galvin Sim Siang 2026¶

Lin Galvin Sim Siang, Hisham Abdul Rauf Badrul, Majid Muhammad Nazmi Abdul, Foong Chan Choong, Ho Ting Khee, Friedlander Lara T. (2026). Refining competency benchmarks: a scoping review of Angoff standard-setting in dental education. BMC Oral Health. https://doi.org/10.1186/s12903-026-08195-9

q2 · i? · review · r2

The review reports from Kim and Yang's comparison of Yes/No versus Percentage rating methods within the Modified Angoff that "the ICC ranging from 0.48 to 0.65" for percentage ratings versus a maximum ICC of 0.41 for Yes/No, making the percentage method preferable.

"However, using percentage ratings in the Modified Angoff method produced higher reliability, with the ICC ranging from 0.48 to 0.65, whereas the ‘Yes/No’ option in the Angoff method had a maximum ICC of 0.41 [34]."

Discussion¶

Related Claims¶

• Angoff-derived cut-off scores exceeded the arbitrary 60% fixed passing mark, with one study reporting a 74.31% cut-off that would have produced a 41% failure rate — related
• Angoff and Modified Angoff standard-setting show good inter-rater reliability in dental education, with reported coefficients ranging from 0.65 to 0.94 — related
• The Borderline Regression method produced higher OSCE pass rates than Angoff methods (97.5% vs 86.6%) and discrepancies raise concerns for small cohorts — related
• The Angoff method produced lower cut-off scores and higher pass rates than Cohen and Hofstee methods in dental written assessments, especially in early and middle years — related
• Contextual factors (learner level, discipline, format) moderate Angoff outcomes — related
• Angoff methods produce significantly higher pass rates than conventional fixed standard-setting methods — related
• Angoff (Yes/No) variant shows the highest variability in cut scores — related
• Modified Angoff with reality check achieves excellent inter-rater reliability — related