Tailor Angoff implementation to programme and assessment needs, using trained multi-disciplinary panels with consensus-building¶

Strategy · All strategies
Evidence · 5 claims (4 for, 1 mixed) · 1 study (1 review), q2 · 0 of 1 report an effect size · 5 claims rest on one study

Description¶

The review recommends that dental educators tailor standard-setting implementation to programme and assessment needs rather than adopting a one-size-fits-all approach. It is "highly recommended that a multi-disciplinary panel be used to reduce subjectivity in the standard-setting process", since panellists with varying specialisations established minimal variations in cut-off scores. Implementation should include panel training to conceptualise the borderline candidate, discussions and consensus-building to mitigate biases, careful selection of response format, and validation of all cut-off scores, as small deviations can greatly affect student outcomes.

Design Implications¶

Context¶

Requirements¶

• Panel training in standard-setting and in conceptualising the borderline candidate
• Discussions and consensus-building among panellists
• Careful review and validation of all assessment items before administration

Constraints¶

• The standard-setting process can be time-consuming due to frequent meetings, and is often complicated by concerns about confidentiality; the literature offers no definitive answer on panel size, with a suggested range of five to thirty

Target Learners¶

• undergraduate dental students

Target Learning Goals¶

• defensible, reliable cut-off scores in summative dental assessments

Claims¶

• Angoff-derived cut-off scores exceeded the arbitrary 60% fixed passing mark, with one study reporting a 74.31% cut-off that would have produced a 41% failure rate [+W]
• Angoff and Modified Angoff standard-setting show good inter-rater reliability in dental education, with reported coefficients ranging from 0.65 to 0.94 [+W]
• The Angoff method produced lower cut-off scores and higher pass rates than Cohen and Hofstee methods in dental written assessments, especially in early and middle years [+W]
• The Borderline Regression method produced higher OSCE pass rates than Angoff methods (97.5% vs 86.6%) and discrepancies raise concerns for small cohorts [~W]
• Percentage rating format in the Modified Angoff method yields higher reliability than the Yes/No format, with ICCs of 0.48–0.65 versus a maximum of 0.41 [+W]

Related Strategies¶

• Adopt structured reporting standards for prescribing assessment research including reliability, validity, cut scores and costs
• Tailor active methodologies to the specific educational context and subject matter

Examples¶

-

Key Sources¶

• Lin Galvin Sim Siang, Hisham Abdul Rauf Badrul, Majid Muhammad Nazmi Abdul, Foong Chan Choong, Ho Ting Khee, Friedlander Lara T. (2026). Refining competency benchmarks: a scoping review of Angoff standard-setting in dental education. BMC Oral Health. https://doi.org/10.1186/s12903-026-08195-9