Expert graders show high agreement among themselves when scoring a written exam on first degree equations¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2013 · checked against its sources 2026-10-29 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Three experts' scores of the same written exam showed a high degree of agreement with one another. → Millán-Valldeperas 2013
Evidence¶
Millán-Valldeperas 2013¶
Millán-Valldeperas, E.; Descalço, L.; Castillo, G.; Oliveira, P.; Diogo, S. (2013). Using Bayesian networks to improve knowledge assessment. Elsevier. http://hdl.handle.net/10630/7068
q2 · i? · design · r2
In the evaluation, the written exam "was graded by three experts", providing the expert-average criterion against which both BSMs were compared. The article states there was a high degree of agreement among the experts' scores; no agreement coefficient value is printed.
"The written exam was graded by three experts. Then two BSMs were developed, one for the computer test and another one for the written exam."
Discussion¶
Related Claims¶
- Diagnosis capability of domain-specific Bayesian student models was tested with 152 students across 12 first-degree-equation concepts — related
- A Bayesian student model's diagnoses agree highly with expert averages on a written exam but poorly on the computerized test — related
- Three independent expert instructors reached exceptionally high inter-rater reliability when grading 1200 bash exam responses, establishing a reliable human reference standard — related
- Expert human raters established an inter-rater reliability benchmark above ICC 0.89 on all four IELTS dimensions for the essay corpus — related
- The framework shows high scoring consistency, with 79% agreement and 81% inter-rater reliability on physics questions — related
- Human inter-rater agreement for FRISCO essay scoring was reliable, with mean kappa 0.78–0.84 and percentage of agreement above 75% in all six dimensions — related
- Human Graders of Open-Ended Work Agree With Each Other Less Than Treating One Grade as Ground Truth Implies — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
expert-inter-rater-agreement-high-written-exam |
title |
Expert graders show high agreement among themselves when scoring a written exam on first degree equations |
description |
Expert graders show high agreement among themselves when scoring a written exam on first degree equations |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2013 |
source_checked |
2026-10-29 |
generated |
by process:wiki-ingest on 2026-10-29 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — millán-valldeperas-2013 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.