Performance-based assessments score low on generalizability, reliability, and standardization, with rater effects influencing scores¶
Claim · All claims
Evidence · 1 study · designr2·q1
Currency · newest study 2018 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2021: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q1 i? PBAs score low on generalizability, reliability, and standardization, and rater effects can influence students' scores, motivating MBPA as an alternative. → Sebastiaan de Klerk 2018
Evidence¶
Sebastiaan de Klerk 2018¶
Sebastiaan de Klerk, Bernard P. Veldkamp, Theo J. H. M. Eggen. (2018). A framework for designing and developing multimedia-based performance assessment in vocational education. Education Tech Research Dev (2018) 66:147–171. https://doi.org/10.1007/s11423-017-9559-5
q1 · i? · design · r2
The article's own assertion in its introduction, citing prior literature (Kane 1990; Linn et al. 1991; Ruiz-Primo et al. 1993; Shavelson et al. 1993, among others) rather than reporting new data. The article offers this as motivation for MBPA as "an alternative and improved approach to the measurement of vocational skills."
"it has often been demonstrated that PBAs score low on gener- alizability, reliability, and standardization, and that rater effects can influence students’ scores"
Discussion¶
Related Claims¶
- Performance assessments can be reliably scored when designed appropriately and used with other measures — reports the opposite
- Making trustworthiness metrics and visualizations explicit increased inter-rater reliability among learning engineers evaluating LLM responses — related
- In one community-college lab class, six scorers rating six teams with an engineering rubric differed by a mean of 1.2 of 9 points, which the authors read as a sign the rubric may not be reliable — related
- The Procedure Based Assessment shows poor engagement and deficient orthopaedic utility evidence, and the authors propose it is no longer appropriate for summative assessment — related
Page metadata
| Field | Value |
|---|---|
status |
review |
generated |
by process:wiki-ingest on 2026-10-28 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.