Global rating scales discriminate between performance levels without a ceiling effect, making them useful for high-stakes summative assessment of advanced residents¶
Claim · All claims
Evidence · 1 study · reviewr3·q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2023: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Global rating scales can discriminate between competent and expert performance, have demonstrated content and concurrent validity and reliability, and their absence of a ceiling effect makes them useful for high-stakes summative assessment. → James H.K. 2020
Evidence¶
James H.K. 2020¶
James H.K., Chapman A.W., Pattison G.T.R., Fisher J.D., Griffin D.R. (2020). Analysis of Tools Used in Assessing Technical Skills and Operative Competence in Trauma and Orthopaedic Surgical Training: A Systematic Review. JBJS Reviews. https://doi.org/10.2106/JBJS.RVW.19.00167
q2 · i? · review · r3
The review's narrative synthesis of global rating scale studies (19 studies). It reports demonstrated content and concurrent validity and reliability, but notes they require expert surgeon evaluators, are time-consuming, and may be susceptible to assessor bias.
"The ability of global rating scales to distinguish between all levels of performance and the absence of a ceiling effect make them useful for high-stakes, summative assessment1 and the assessment of advanced residents."
Discussion¶
Related Claims¶
- The 105 included studies of orthopaedic technical-skills assessment were predominantly conducted in simulated settings with 2,088 resident participants
- The Procedure Based Assessment shows poor engagement and deficient orthopaedic utility evidence, and the authors propose it is no longer appropriate for summative assessment
- Checklists show reasonable validity and reliability but have an early ceiling effect and are most appropriate for novice learners in formative settings — related
- No current technical skills-assessment tool in orthopaedic surgical training fulfills accepted criteria for effective assessment — related
- Global rating scales by expert practitioners are as reliable as item-specific checklists and less labour intensive to devise — related
Page metadata
| Field | Value |
|---|---|
status |
review |
generated |
by process:wiki-ingest on 2026-11-01 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.