Structured criteria in peer assessment are not shown to improve its effect on academic performance over freeform review¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q4 · i0 negligible · n=54 studies, k=141 effect sizes
Currency · newest study 2020 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

The one meta-analysis recorded here (Double et al. 2020, 54 control-group studies) found that giving peer reviewers a specific script or rubric did not significantly change peer assessment's effect on academic performance compared with freeform review (structured g = 0.29, freeform g = 0.42, difference not significant; the freeform subgroup is small). Whether structured criteria make peer judgments more reliable is a separate question this evidence does not test. The proposed mechanism, that criteria direct reviewers to specific dimensions of quality rather than general impressions, remains a hypothesis here.

Subclaims¶

q4 i0 A meta-analysis of 54 studies (k=141) of peer assessment and academic performance found no significant difference between studies giving reviewers a specific script/rubric (g=0.29) and freeform, unstructured peer assessment (g=0.42) — the moderator effect for structure was small and non-significant (b=-0.13, 95% CI -0.51 to 0.25, p=.455), so this synthesis does not support that structured criteria improve outcomes over open-ended review, though the freeform subgroup was small (9 studies, k=29). → Double et al. 2020

Evidence¶

Double et al. 2020¶

Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The impact of peer assessment on academic performance: A meta-analysis of control group studies. Educational Psychology Review, 32(2), 481–509. doi:10.1007/s10648-019-09510-3

q4 · meta-analysis of experimental/quasi-experimental control-group studies · i0 · moderator effect below d=0.2, non-significant · n=54 studies, k=141 effect sizes · quant-synthesis · r2

Meta-analysis of 54 experimental/quasi-experimental studies (k=141 effect sizes) on peer assessment's effect on academic performance in primary, secondary and tertiary students. Overall peer assessment showed a small-to-medium benefit across comparison groups (g=0.31, p<.001), and versus no assessment specifically g=0.31, p=.004. Among several moderators tested via meta-regression, one directly addresses this claim: whether a specific script or rubric was provided to reviewers versus freeform, unstructured assessment. Structured-criteria studies showed g=0.29 (p<.001) and freeform studies showed a numerically larger g=0.42 (p=.030), but the difference between them was small and not statistically significant (b=-0.13, 95% CI -0.51 to 0.25, p=.455), and the authors note the freeform subgroup was small (9 studies, k=29, against 45 studies, k=112, for structured assessment). This is the only outcome-level moderator test of criteria structure in the corpus and it does not support the claim as stated — see Discussion.

Note for whoever merges this: this evidence qualifies rather than confirms the page's claim — the meta-analysis's own moderator test runs numerically opposite to "structured beats unstructured" (freeform g=0.42 > structured g=0.29), though non-significant and based on a small freeform subgroup (9 studies, k=29). It also tests structure's effect on the academic performance of students who, in most included studies, both assessed and were assessed (the meta-analysis cannot separate the two roles), not on inter-rater reliability — a claim about rubric effects on scoring reliability is a different construct and is not addressed here.

Discussion¶

The recorded evidence runs against the title. In the one meta-analysis recorded above, studies using structured criteria (rubrics or scripts) showed g = 0.29 and freeform studies g = 0.42; the difference was not significant, and the freeform subgroup is small (9 studies, k=29). So structured criteria are not shown to improve the learning effect of peer assessment. Their benefit for the reliability of peer judgments is a separate question this evidence does not test. The page was retitled on 2026-09-30 to say so.

Why structure matters. Unstructured peer review invites halo effects, leniency bias, and attention to surface features (length, neatness) rather than quality of reasoning. Structured criteria direct reviewers' attention to specific dimensions of quality, which benefits reviewers as much as reviewees: applying criteria to a peer's work is itself an act of evaluation and self-monitoring that can transfer to revising one's own work. This aligns with the broader logic of Assessment for learning improves achievement — assessment tasks that make quality criteria visible support learning, not just grading.

Boundary conditions. Structured criteria may backfire: a checklist-only focus could produce mechanical, criterion-ticking reviews with little substantive feedback. That is a hypothesis; no evidence on this page tests it. (The related claim on one-session checklist (CRAAP) instruction in source evaluation concerns teaching students to evaluate online sources, not rubrics in peer assessment, so it does not bear on it.) Criteria also cannot compensate for reviewers who lack the domain knowledge to recognize quality; novices may apply criteria literally without understanding the underlying standard. Training reviewers on worked examples of graded work before assessing, and combining criteria with open comment fields, are common mitigations.

Moderators to watch. The value of structure likely depends on (a) reviewer expertise — highly detailed rubrics may lose value or even hinder learners with more developed evaluative skill, consistent with the expertise reversal effect; (b) stakes — criteria may improve reliability most when reviews carry grading consequences; and (c) reviewer training — criteria without calibration practice may still yield inconsistent judgments.

Open questions. How much structure is optimal — detailed rubrics versus a few holistic criteria — and how criteria interact with anonymity, grading stakes, and reviewer expertise remain under-specified.

Related Claims¶

• Self-assessing against a rubric improved the quality of student work in two non-randomised studies, one of which also found more stress and performance-avoidance — the general claim this finding bears on, with the studies behind it

• Assessment for learning improves achievement — structured peer assessment is a form of assessment that feeds learning rather than merely certifying it

• One-session checklist (CRAAP) instruction in source evaluation did no better than the alternative methods it was compared with in two quasi-experiments — related: a checklist taught for evaluating online sources, not a peer-assessment rubric; it does not test whether rubrics breed compliance
• Self-regulated learning — evaluating peers against criteria exercises the self-monitoring and judgment central to self-regulation
• Expertise reversal effect — highly structured supports like detailed rubrics can lose value or hinder learners with more developed evaluative skill
• Online peer assessment shifts instructional assessment from teacher-controlled to a combination of teacher assessment and student peer assessment — related
• Peer assessment improves academic performance by a small average amount relative to no assessment or to teacher assessment — related
• In two secondary-school writing studies, peer reviewers' own performance predicted how helpful their feedback was but not how accurately it identified problems — related
• In two online comparisons, a rubric changed what peer reviewers commented on (one study) and helped peer raters place writing in quality tiers, most for complex criteria (the other); the first study says the rubric restrained reflective thinking — related
• Peer and self-assessment both show significant effects on academic performance with no significant difference between their impacts — related
• The Trainer is expected to make peer comments more specific, criterion-referenced, and evidence-based — related
• The benefits of rubrics on self-assessment accuracy do not depend on the inferential level of the assessment criteria — related
• Providing a rubric does not significantly improve relative accuracy of self-assessment across criteria — related
• Most moderator variables are not significantly related to rubric effects on academic performance, including rubric design specifics — related
• Using rubrics has a positive, moderate effect on students' academic performance (corrected g = 0.45; standardized mean change = 0.38) — related
• Quality control in peer assessment can be exercised proactively or reactively on students and their assessments — related
• Structured Peer Tutoring Outperforms Unstructured — reports the opposite
• Peer Feedback Improves Writing — related
• For self-regulated learning outcomes, rubrics combined with both self- and peer assessment show a significantly larger effect than rubrics with self-assessment alone — related
• Holistic peer feedback correlated significantly with tutor marks while specific-criteria rubric feedback did not in the PEARS system — related
• Most studies used online peer feedback to improve students' task performance — related
• Rubrics and training were the most frequently implemented presentation modes of online peer feedback — related