Peer marks and grades show acceptably high reliability with teachers' marks in most comparison studies, but a substantial minority report low reliability¶

Claim · All claims
Evidence · 1 study (2 entries) · review r3 · q2
Currency · newest study 1998 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Of 25 studies comparing teacher and peer marks or grades, 18 (72%) reported acceptably high reliability. → Keith Topping 1998 q2 i? A substantial minority of studies (7) found the reliability and validity of peer assessment unacceptably low in particular projects. → Keith Topping 1998 (2)

Evidence¶

Keith Topping 1998¶

Keith Topping. (1998). Peer Assessment Between Students in Colleges and Universities. Review of Educational Research. https://doi.org/10.3102/00346543068003249

q2 · i? · review · r3

Narrative review synthesis of 31 reliability studies in higher education, most focusing on scores and grades. The review reports "18 (72%) reported acceptably high reliability", sometimes with coefficients such as r = .88, and notes peer marks sometimes clustered around the median.

"Of 25 studies comparing teacher and peer marks or grades, 18 (72%) reported acceptably high reliability, often expressed in correlation coefficients, percentage agreement, or measures of central tendency and variance, sometimes with indication of statistical significance (e.g., r - .88, Hughes & Large, 1993a, 1993b)."

Keith Topping 1998 (2)¶

Keith Topping. (1998). Peer Assessment Between Students in Colleges and Universities. Review of Educational Research. https://doi.org/10.3102/00346543068003249

q2 · i? · review · r3

The review's summary of its 31 reliability/validity studies: a majority support adequacy, but "a substantial minority (7)" found unacceptably low reliability in particular projects, including essay writing, oral presentations and group-project contributions. Low-reliability studies may be less likely to be published.

"The majority of studies (18) suggest that peer assessment is of adequate reliability and validity in a wide variety of applications. However, a substantial minority (7) found the reliability and validity of peer assessment unacceptably low in particular projects."

Discussion¶

Related Claims¶

• Human-LLM agreement remains low across constructs (κ = 0.000 to 0.419), with self-efficacy showing the best alignment and Prior KSAs none — related
• Human-LLM agreement is moderated by code properties, multi-model consensus, and model-reported confidence — related
• Peer assessment through tests, marks, and grades produces learning gains and reduced anxiety, with most participants reporting learning benefits — related
• Peer assessment of professional skills shows adequate reliability including in medicine, with outcomes generally at least equivalent to teacher assessment — a narrower finding that bears on this claim
• Grading students on the quality of their reviews rather than on peer-assessed marks reduced tensions and improved student-instructor mark correlations — related
• Peer assessment is equivalent to teacher assessment for exact-science exams with clear marking schemes — a narrower finding that bears on this claim
• Peer assessment of writing yields outcomes at least as good as teacher assessment and sometimes better — related
• An accountability feature allowing students to grade received reviews improved the correlation between expert and peer marks in peerScholar — a narrower finding that bears on this claim
• The Bell Chemistry Test correlates moderately with semester marks and teachers' estimates, and is judged a fairly good measure of class knowledge but unreliable for an individual. — related