Claims

In one community-college lab class, six scorers rating six teams with an engineering rubric differed by a mean of 1.2 of 9 points, which the authors read as a sign the rubric may not be reliable

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2010 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2013: tool and model accuracy changes quickly

Subclaims

Reading the evidence codes

q is the design tier, i the impact, n the sample, and the kind with its r the rigour judged within that kind: what each code means.

This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.

q2 i? Scoring six teams with the engineering rubric gave a mean class score of 6.2 of 9 points and a mean class standard deviation of 1.2 points, leading the authors to conclude the rubric may not have been reliable. → Morzinski 2010

Evidence

Morzinski 2010

Morzinski, K., Azucena, O., Downs, C., Favaloro, T., Park, J., & U, V. (2010). Circuit Design: An inquiry lab activity at Maui Community College. ASP Conference Series. https://arxiv.org/abs/1009.3296 doi:10.48550/arxiv.1009.3296

q2 · i? · design · r2

Analysis of summative assessment scores in which six scorers rated six teams using the rubric (Table 7); the article prints the mean class score of 6.2 points and the mean class standard deviation of 1.2 points, "or 13% out of 9 points." No effect size is reported.

"The mean class standard deviation was 1.2 points, or 13% out of 9 points. This is more than one point uncertainty, implying that the rubric may not have been a reliable test and can be improved."

Discussion

The one entry is a single class at Maui Community College: six scorers, six teams, one inquiry lab. No reliability coefficient (such as an ICC or kappa) is reported; the figure is a mean standard deviation across scorers, and "may not have been a reliable test" is the authors' own reading of it. Whether the rubric, the scorers' training or the small number of teams is responsible cannot be told from this, and nothing here generalises to engineering rubrics as such.

Page metadata
Field Value
type claim
id engineering-rubric-low-scoring-reliability
title In one community-college lab class, six scorers rating six teams with an engineering rubric differed by a mean of 1.2 of 9 points, which the authors read as a sign the rubric may not be reliable
description In one community-college lab class, six scorers rating six teams with an engineering rubric differed by a mean of 1.2 of 9 points, which the authors read as a sign the rubric may not be reliable
status review
evidence_strength weak
newest_evidence 2010
source_checked 2026-10-20
recheck_by 2013
generated by process:wiki-ingest on 2026-09-26
trust unverified — no one has confirmed this page's content
sources 1 — morzinski-2010

Frontmatter is the page's machine-readable record — see the Schema & Guide.