Item analysis flags items whose marking instructions mislead peer assessors¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2009 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? On CA-2007 question Q7, peers were far more severe (p=0.74) than the teacher (p=0.59), traced to marking instructions that did not preserve independence of the marking scheme's parts; Q7 was excluded from the validity analysis. → Bouzidi 2009
Evidence¶
Bouzidi 2009¶
Bouzidi, L., & Jaillet, A. (2009). Can Online Peer Assessment be Trusted?. Educational Technology & Society, 12(4), 257-268. https://www.ifets.info/
q2 · i? · design · r2
Item analysis of the CA-2007 exam (13 items, 94 sheets): difficulty indexes for Q7 diverged between peers and teacher, and the authors attribute this to "a poor indication of the marking instructions for this item", so Q7 was disregarded in the validity analysis.
"the students marked the answers to question Q7 severely (p=0.74), whereas the teacher marked the answers to the same question with average severity (p=0.59). The difference is too large to be explained by chance alone."
Discussion¶
Related Claims¶
- Peer assessment is equivalent to teacher assessment for exact-science exams with clear marking schemes — related
- Peers slightly under-mark their comrades' papers while self-assessors over-mark their own — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
item-analysis-flags-marking-instruction-problems |
title |
Item analysis flags items whose marking instructions mislead peer assessors |
description |
Item analysis flags items whose marking instructions mislead peer assessors |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2009 |
generated |
by process:wiki-ingest on 2026-10-16 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — bouzidi-2009 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.